-- Logs begin at Thu 2022-08-04 06:44:58 UTC, end at Thu 2022-08-04 08:07:48 UTC. -- Aug 04 07:22:25.548226 dv-cinder-849800-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 04 07:22:25.558221 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 04 07:22:25.560286 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 4 07:22:25 2022] *** Aug 04 07:22:25.560286 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 04 07:22:25.560286 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 04 07:22:25.560286 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: nodename: dv-cinder-849800-3-iscsi Aug 04 07:22:25.560286 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: machine: x86_64 Aug 04 07:22:25.560286 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: clock source: unix Aug 04 07:22:25.560286 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: pcre jit disabled Aug 04 07:22:25.560286 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: detected number of CPU cores: 4 Aug 04 07:22:25.560286 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: current working directory: / Aug 04 07:22:25.560286 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: detected binary path: /usr/bin/uwsgi-core Aug 04 07:22:25.560286 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: your processes number limit is 47680 Aug 04 07:22:25.560906 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: your memory page size is 4096 bytes Aug 04 07:22:25.560906 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: detected max file descriptor number: 2048 Aug 04 07:22:25.560906 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: lock engine: pthread robust mutexes Aug 04 07:22:25.560906 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: thunder lock: enabled Aug 04 07:22:25.560906 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 04 07:22:25.560906 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 04 07:22:25.599547 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: Python main interpreter initialized at 0x555a330309c0 Aug 04 07:22:25.599547 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: python threads support enabled Aug 04 07:22:25.599547 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: your server socket listen backlog is limited to 100 connections Aug 04 07:22:25.599547 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: your mercy for graceful operations on workers is 90 seconds Aug 04 07:22:25.599895 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: mapped 403077 bytes (393 KB) for 2 cores Aug 04 07:22:25.599971 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: *** Operational MODE: preforking *** Aug 04 07:22:25.600272 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: *** uWSGI is running in multiple interpreter mode *** Aug 04 07:22:25.600272 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: spawned uWSGI master process (pid: 80285) Aug 04 07:22:25.600467 dv-cinder-849800-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 04 07:22:25.602423 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: spawned uWSGI worker 1 (pid: 80297, cores: 1) Aug 04 07:22:25.602423 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: spawned uWSGI worker 2 (pid: 80298, cores: 1) Aug 04 07:22:25.602423 dv-cinder-849800-3-iscsi devstack@keystone.service[80285]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 04 07:22:31.146852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 07:22:31.146852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: from cryptography.utils import int_from_bytes Aug 04 07:22:31.147633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 07:22:31.147633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: from cryptography.utils import int_from_bytes Aug 04 07:22:31.150720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 07:22:31.150720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: from cryptography.utils import int_from_bytes Aug 04 07:22:31.151176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 07:22:31.151176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: from cryptography.utils import int_from_bytes Aug 04 07:22:31.206558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 07:22:31.206755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 07:22:31.206950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] command line args: [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 07:22:31.207118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 07:22:31.207206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 07:22:31.207317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 07:22:31.207402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ================================================================================ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 07:22:31.207514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] command line args: [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 07:22:31.207625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] admin_token = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.207787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 07:22:31.207870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_dir = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.207953 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ================================================================================ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 07:22:31.208058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.208234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] admin_token = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.208342 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_source = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.208420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] debug = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.208498 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_dir = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.208602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.208907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.208907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_source = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.209035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.209100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] debug = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.209171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.209235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.209418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.209522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.209629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.209737 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.209844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.209984 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.210085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_config_append = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.210179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.210286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.210447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.210559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_dir = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.210661 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_config_append = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.210770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.210981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.211114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.211225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_dir = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.211304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.211420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.211420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.211601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.211675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.211797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.211797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.212562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.212650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.212768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.212891 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.212977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.212977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.213254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.213331 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.213448 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.213532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.213610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.213731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.213813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.213884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.213957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.214030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.214149 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.214223 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.214302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.214429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.214518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.214676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.214975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.215087 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.215193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] publish_errors = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.215368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.215481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.215626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.215828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] publish_errors = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.215936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.216090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.216207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.216323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.216432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.216528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.216612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] standard_threads = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.216727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.216799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.216874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.216955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] standard_threads = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_journal = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_json = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217560 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_stderr = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_syslog = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_journal = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_json = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.217994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.218188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_stderr = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.218313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.218433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_syslog = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.218552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.218665 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 07:22:31.218891 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.219011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.219155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.219270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.219382 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.219501 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.219624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.219738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.219852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.219970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.220116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.220865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.220993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.221107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.221223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.221337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.221455 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.221582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.221697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.221869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.221985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.222097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.222216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.222338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.222452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.222587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.222709 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.222823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.222942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.223067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.223181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.223292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.223411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.223524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.223636 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.223755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.223868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.224027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.224155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.224277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.224391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.224512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.224631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.224751 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.224864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.224976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.225090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.225277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.225392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.225511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.225630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.225743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.225862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.225983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.226098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.226210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.226371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.226483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.226596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.226716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.226836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.226949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.227061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.227175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.227286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.227406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.227535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.227679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.227816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.227930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.228072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.228198 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.228313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.228425 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.228537 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.228537 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.228726 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.228825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.228920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.229014 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.229109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.229236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.229349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.229447 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.229543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.229654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.229739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.229849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.229970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.230086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.230200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.230326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.230441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.230553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.230674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.230800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.230916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.231037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.231152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.231269 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.231390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.231505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.231626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.231732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.231850 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.231963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.232133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.232257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.232373 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.232486 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.232599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.232719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.232832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.232953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.233074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.233188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.233290 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.233404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.233517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.233517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.233739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.233858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.233981 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.234096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.234209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.234329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.234444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.234566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.234681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.234800 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.234914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.235026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.235150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.235272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.235386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.235500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.235612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.235725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.235846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.235967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.236112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.236227 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.236340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.236452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.236637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.236760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.237131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.237253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.237367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.237480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.237601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.237716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.237829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.237942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.238061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.238181 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.238308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.238438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.238602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.238717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.238831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.238950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.239065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.239190 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.239305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.239418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.239537 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.239651 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.239758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.239878 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.240011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.240127 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.240240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.240353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.240476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.240599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.240714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.240827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.240940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.241052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.241165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.241291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.241406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.241518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.241630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.241749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.241864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.241985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.242099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.242212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.242324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.242436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.242548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.242669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.242790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.242904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.243016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.243159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.243294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.243417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.243532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.243645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.243762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.243883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.244046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.external = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.244176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.244292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.244405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.244517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.244621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.external = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.244741 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.244886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.245012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.password = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.245135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.245257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.token = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.245372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.245484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.245605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.password = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.245720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.245841 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.token = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.245956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.246070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.246183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.246296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.246409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.246531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.246646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.246759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.246871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.246984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.247106 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.247221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.247334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.247446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.247558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.247671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.247796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.247911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.248041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.248376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.248494 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.248607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.248730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.248854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.248969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.249083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.249198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.249311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.249433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.249549 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.249662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.249777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.249890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.250011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.250135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.250252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.250365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.250487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.250601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.250714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.250902 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.251020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.251134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.251252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.251365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.251479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.251601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.251718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.251831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.251945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.252089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.252210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.252335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.252451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.252566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.252680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.252793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.252906 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.253029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.253145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.253259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.253374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.253488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.253601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.253724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.253840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.253954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.254067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.254181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.254294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.254416 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.254532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.254645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.254645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.254860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.254973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.255095 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.255210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.255323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.255437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.255550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.255664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.255786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.255901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.256028 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.256143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.256268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.256384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.256507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.256629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.256751 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.256866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.256988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.257111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.257233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.257348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.257462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.257576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.257690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.257816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.257933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.258047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.258160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.258273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.258387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.258510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.258626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.259232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.259348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.259462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.259575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.259698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.259819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.259933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.260072 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.260189 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.260303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.260426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.260542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.260656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.260769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.260883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.260996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.261118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.261236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.261351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.261464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.261577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.261691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.261813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.261929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.262048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.262162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.262276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.262389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.262510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.262626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.262739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.262860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.262975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.263089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.263210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.263331 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.263446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.263560 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.263673 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.263786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.263913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.264077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.264194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.264307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.264421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.264535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.264661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.264778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.264892 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.265006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.265120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.265233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.265359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.265476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.265590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.265704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.265817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.265932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.266054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.266170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.266286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.266400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.266513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.266627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.266801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.266921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.267036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.267150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.267263 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.267377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.267502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.267618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.267733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.267846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.267960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.268090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.268217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.268391 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.268510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.268625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.268739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.268861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.268978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.269093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.269207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.269320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.269434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.269557 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.269672 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.269786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.269908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.270023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.270138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.270261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.270376 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.270490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.270604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.270717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.270831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.270954 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.271070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.271185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.271305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.271419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.271533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.271655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.271771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.271885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.272054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.272173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.272288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.272414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.272531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.272645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.272758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.272872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.272976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.273097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.273212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.273327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.273447 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.273562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.273676 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.273860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.273977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.274091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.274205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.274320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.275068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.275198 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.275314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.275314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.275530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.275643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.275756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.275879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.276014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.276125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.276222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.276319 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.276435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.276563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.276680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.276793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.276907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.277021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.277134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.277256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.277373 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.277487 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.277607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.277723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.277836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.277958 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.278073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.278187 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.278187 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.278400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.278515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.278637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.278752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.278866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.278866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.279079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.279079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.279300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.279300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.279514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.279629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.279748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.279869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.280010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.280010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.280229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.280342 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.280455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.280569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.280695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.280812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.280926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.281040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.281160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.281283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.281431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.281431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.281646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.281760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.281873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.281994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.282110 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.282225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.282338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.282451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.282626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.282626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.282882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.282995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.283108 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.283222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.283335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.283457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.283573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.283686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.283801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.283914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.284068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.284197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.284313 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.284427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.284540 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.284654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.284767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.284889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.285005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.285118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.285231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.285335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.285448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.285575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.285690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.285804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.285917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.286031 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.286144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.286266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.286381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.286494 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.286607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.286721 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.286834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.286957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.287072 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.287185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.287298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.287411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.287533 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.287648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.287761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.287874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.288002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.288118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.288243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.288360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.288473 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.288586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.288700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.288813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.288935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.289051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.289165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.289279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.289393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.289507 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.289628 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.289743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.289857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.289995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.290116 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.290229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.290355 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.290478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.290601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.290869 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.290996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.291112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.291226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.291340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.291467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.291584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.291698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.291821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.291945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.292083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.292199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.292326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.292443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.292443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.292658 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.292771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.292885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.293073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.293194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.293308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.293423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.293537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.293652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.293774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.293890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.294004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.294118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.294233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.294347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.294469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.294585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.294699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.294813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.294813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 07:22:31.295027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.295027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.295027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.295027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.295027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 07:22:31.295027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 07:22:31.322417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.322644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.322871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.323519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.323756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.324002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.331191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.332349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.361404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.361630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.362858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.363095 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.414800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.416065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.465914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.466060 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.467318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.467557 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.530705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.530935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.531159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.531354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.531459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.531582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.531665 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.531797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.531881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.532039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.532263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.532483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.554581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.554815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.554949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.555035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.555183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.555265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.555410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.555493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.556242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.556349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.556448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.556522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.556795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.557100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.567522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.568121 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.580778 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.581010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.581235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.581428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.581513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.581663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.581744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.581874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.581957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.582085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.582168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.582304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.582526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.582744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 07:22:31.599510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.599725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 07:22:31.599808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.599964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 07:22:31.600128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.600281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 07:22:31.600361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.600508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 07:22:31.600588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.600776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.600866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.601032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.601112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.601283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.601359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.601532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.611054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.611219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.611331 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.611496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.611602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.611763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.612167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.612325 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.619648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.619855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.624487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.624622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.624767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.624912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.625049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.625167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.625329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.625445 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.625605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.625720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.625859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.625974 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.626133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.626223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.626409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.626488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.626677 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.626758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.626985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.627064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.627244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.627336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.650912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.651891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.652471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.652686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.653411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.653689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.657333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.657612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.657889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.658394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.658674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.658956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.664531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.664810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.665972 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.666253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.669152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.669436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.669726 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.670014 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.670288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.670450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.670578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.670736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.670852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.671018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.671106 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.671298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.671576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.671855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.672172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.672454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.689002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.689164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.689276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.689439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.693218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.693405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.696995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.697157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.697288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.697449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.701757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.701892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.705226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.705502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.708390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.708696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.708778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.708979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.709082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.709266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.709356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.709547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.709628 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.709820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.709901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.710099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.726155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.726385 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.726469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.726652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.726739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.726914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.726996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.727193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.734137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.734792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.735697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.735970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.736356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.736632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.739081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.739361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.739649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.739728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.739927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.740036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.740234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.740316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.740569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.740848 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.752322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.752595 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.752870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.753142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.753223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.753415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.753511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.753693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.753773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.753969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.754049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.754243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.754525 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.754814 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.770480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.771295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.773723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.774345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.777526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.777746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.780036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.780267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.781589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.781763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.781871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.782048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.782160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.782338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.788671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.788845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.790410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.790857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.793433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.793722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.793954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.794035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.794240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.794320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.794519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.794796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.802404 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.802693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.802871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.802978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.803161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.803438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.810574 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.810854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.810993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 07:22:31.811132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.811270 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.811406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.811556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.811676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.811830 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.811944 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.812124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.812238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.812395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.812516 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.812665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.812903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.812983 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.813188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.813313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.813459 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.813587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.813723 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.813859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.813987 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.814132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.814254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.814398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.814667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 07:22:31.848474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555a330309c0 pid: 80297 (default app) Aug 04 07:22:31.848573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555a330309c0 pid: 80298 (default app) Aug 04 07:22:31.851758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ae1bc-ecf9-4599-8c33-bf23bbb5751e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:31.852083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ae1bc-ecf9-4599-8c33-bf23bbb5751e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:31.852365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ae1bc-ecf9-4599-8c33-bf23bbb5751e None None] PATH_INFO: `/v3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:31.856778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1/1] 10.222.0.64 () {50 vars in 791 bytes} [Thu Aug 4 07:22:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:22:40.407739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e3bf8-c215-4579-a843-62e4bb77b21b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:40.408254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e3bf8-c215-4579-a843-62e4bb77b21b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:40.408254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e3bf8-c215-4579-a843-62e4bb77b21b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:40.412859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1/2] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:22:40.418612 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c90235-7d02-46e2-99ab-159334a9a8b0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:40.419004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c90235-7d02-46e2-99ab-159334a9a8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:40.419162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c90235-7d02-46e2-99ab-159334a9a8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:40.489050 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: INFO dbcounter [None req-d8c90235-7d02-46e2-99ab-159334a9a8b0 None None] Registered counter for database keystone Aug 04 07:22:40.508967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG oslo_db.sqlalchemy.engines [None req-d8c90235-7d02-46e2-99ab-159334a9a8b0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80297) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 07:22:40.515123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writer thread running {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 07:22:40.692082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG passlib.handlers.bcrypt [None req-d8c90235-7d02-46e2-99ab-159334a9a8b0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80297) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 07:22:40.692492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG passlib.handlers.bcrypt [None req-d8c90235-7d02-46e2-99ab-159334a9a8b0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80297) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 07:22:40.747367 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d8c90235-7d02-46e2-99ab-159334a9a8b0 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:40.748973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d8c90235-7d02-46e2-99ab-159334a9a8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:40.877088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 2/3] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 07:22:41.028548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e3bf8-c215-4579-a843-62e4bb77b21b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:41.098358 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: INFO dbcounter [None req-267e3bf8-c215-4579-a843-62e4bb77b21b None None] Registered counter for database keystone Aug 04 07:22:41.119735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG oslo_db.sqlalchemy.engines [None req-267e3bf8-c215-4579-a843-62e4bb77b21b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80298) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 07:22:41.125900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writer thread running {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 07:22:41.243248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e3bf8-c215-4579-a843-62e4bb77b21b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:41.247490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c8271-b676-4137-b1c4-61077501c2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:41.248147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c8271-b676-4137-b1c4-61077501c2fa admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:41.248434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c8271-b676-4137-b1c4-61077501c2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:41.248711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c8271-b676-4137-b1c4-61077501c2fa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:41.256745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c8271-b676-4137-b1c4-61077501c2fa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:41.309696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0e0c8271-b676-4137-b1c4-61077501c2fa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 07:22:41.310042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c8271-b676-4137-b1c4-61077501c2fa admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:41.319507 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 2/4] 10.222.0.64 () {58 vars in 1324 bytes} [Thu Aug 4 07:22:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 07:22:41.326071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c90235-7d02-46e2-99ab-159334a9a8b0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:41.340304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c90235-7d02-46e2-99ab-159334a9a8b0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:41.344709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8522c-f499-4f02-95d6-a2972fd8a545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:41.345810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8522c-f499-4f02-95d6-a2972fd8a545 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:41.346097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8522c-f499-4f02-95d6-a2972fd8a545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:41.346373 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8522c-f499-4f02-95d6-a2972fd8a545 admin admin] PATH_INFO: `/v3/projects/60367118cabe491a9e4ea2e0f6945aa3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:41.347823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8522c-f499-4f02-95d6-a2972fd8a545 admin admin] RBAC: Authorizing `identity:get_project(project_id=60367118cabe491a9e4ea2e0f6945aa3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:41.398988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f3a8522c-f499-4f02-95d6-a2972fd8a545 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 07:22:41.399324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8522c-f499-4f02-95d6-a2972fd8a545 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:41.403856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 3/5] 10.222.0.64 () {56 vars in 1207 bytes} [Thu Aug 4 07:22:41 2022] GET /identity/v3/projects/60367118cabe491a9e4ea2e0f6945aa3 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:22:43.374995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b71c7-56ad-4547-b89d-1c2443d4117a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:43.375520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b71c7-56ad-4547-b89d-1c2443d4117a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:43.375520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b71c7-56ad-4547-b89d-1c2443d4117a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:43.379599 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 3/6] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:43.384889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177317b-8a06-496f-954c-714734759930 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:43.385276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177317b-8a06-496f-954c-714734759930 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:43.385443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177317b-8a06-496f-954c-714734759930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:43.405015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1177317b-8a06-496f-954c-714734759930 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:43.406510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1177317b-8a06-496f-954c-714734759930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:43.414799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 4/7] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 04 07:22:43.436895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b71c7-56ad-4547-b89d-1c2443d4117a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:43.449203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b71c7-56ad-4547-b89d-1c2443d4117a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:43.453438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5bd3f-f7a3-47d2-ab04-861397c01f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:43.454045 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5bd3f-f7a3-47d2-ab04-861397c01f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:43.454332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5bd3f-f7a3-47d2-ab04-861397c01f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:43.454615 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5bd3f-f7a3-47d2-ab04-861397c01f23 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:43.462414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5bd3f-f7a3-47d2-ab04-861397c01f23 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:43.464231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5bd3f-f7a3-47d2-ab04-861397c01f23 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:43.472936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 4/8] 10.222.0.64 () {58 vars in 1324 bytes} [Thu Aug 4 07:22:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:22:43.479140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177317b-8a06-496f-954c-714734759930 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:43.488241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177317b-8a06-496f-954c-714734759930 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:43.493161 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72c9f9-5a7f-45d7-aebc-af71bb58931a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:43.493971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72c9f9-5a7f-45d7-aebc-af71bb58931a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:43.494251 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72c9f9-5a7f-45d7-aebc-af71bb58931a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:43.494522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72c9f9-5a7f-45d7-aebc-af71bb58931a admin admin] PATH_INFO: `/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:43.495863 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72c9f9-5a7f-45d7-aebc-af71bb58931a admin admin] RBAC: Authorizing `identity:get_user(user_id=79d0d7df7a094b5ba4b9a9bb3b449967)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:43.497693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72c9f9-5a7f-45d7-aebc-af71bb58931a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:43.499197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 5/9] 10.222.0.64 () {56 vars in 1198 bytes} [Thu Aug 4 07:22:43 2022] GET /identity/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:22:45.567672 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53291-c6a8-485f-8a55-638732cf6d9d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:45.568241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53291-c6a8-485f-8a55-638732cf6d9d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:45.568241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53291-c6a8-485f-8a55-638732cf6d9d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:45.569577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 5/10] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:45.572263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d289e8-f689-460f-9052-fee61d715536 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:45.572686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d289e8-f689-460f-9052-fee61d715536 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:45.572827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d289e8-f689-460f-9052-fee61d715536 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:45.574141 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 6/11] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:45.574996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe444d7-8c97-4cfb-bafb-83b77bb9ccb8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:45.575285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe444d7-8c97-4cfb-bafb-83b77bb9ccb8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:45.575566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe444d7-8c97-4cfb-bafb-83b77bb9ccb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:45.579092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae7d06-815e-40a9-bbdd-af2d22560a52 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:45.579384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae7d06-815e-40a9-bbdd-af2d22560a52 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:45.579666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae7d06-815e-40a9-bbdd-af2d22560a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:45.598783 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-79ae7d06-815e-40a9-bbdd-af2d22560a52 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:45.599966 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-79ae7d06-815e-40a9-bbdd-af2d22560a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:45.617953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG passlib.handlers.bcrypt [None req-ebe444d7-8c97-4cfb-bafb-83b77bb9ccb8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80298) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 07:22:45.618371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG passlib.handlers.bcrypt [None req-ebe444d7-8c97-4cfb-bafb-83b77bb9ccb8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80298) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 07:22:45.664354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ebe444d7-8c97-4cfb-bafb-83b77bb9ccb8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:45.665978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ebe444d7-8c97-4cfb-bafb-83b77bb9ccb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:45.679182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 7/12] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:22:45.701435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae7d06-815e-40a9-bbdd-af2d22560a52 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:45.725096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:45.767226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 6/13] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:22:45.792887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe444d7-8c97-4cfb-bafb-83b77bb9ccb8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:45.805365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae7d06-815e-40a9-bbdd-af2d22560a52 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:45.838929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43ef45-37be-4711-8720-3562ffcb3b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:45.839476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43ef45-37be-4711-8720-3562ffcb3b0d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:45.839753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43ef45-37be-4711-8720-3562ffcb3b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:45.840048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43ef45-37be-4711-8720-3562ffcb3b0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:45.891045 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe444d7-8c97-4cfb-bafb-83b77bb9ccb8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:45.893519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:45.905905 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43ef45-37be-4711-8720-3562ffcb3b0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:45.907388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43ef45-37be-4711-8720-3562ffcb3b0d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:45.928300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd6e90-088e-4794-9a46-7a7bd873b80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:45.929513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6e90-088e-4794-9a46-7a7bd873b80b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:45.930302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6e90-088e-4794-9a46-7a7bd873b80b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:45.930426 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:45.930790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6e90-088e-4794-9a46-7a7bd873b80b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:45.943039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd6e90-088e-4794-9a46-7a7bd873b80b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c4dd6e90-088e-4794-9a46-7a7bd873b80b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:22:45.951929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:22:45.953078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:22:45.954122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 7/14] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:22:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:22:45.959616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd6e90-088e-4794-9a46-7a7bd873b80b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:46.011263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 8/15] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:22:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 07:22:46.016928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43ef45-37be-4711-8720-3562ffcb3b0d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:46.060878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd6e90-088e-4794-9a46-7a7bd873b80b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:46.094367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b5e0a-06c9-429e-aae4-2940ff4d5a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:46.094908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b5e0a-06c9-429e-aae4-2940ff4d5a6c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:46.095188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b5e0a-06c9-429e-aae4-2940ff4d5a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:46.095460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b5e0a-06c9-429e-aae4-2940ff4d5a6c None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:46.096205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b5e0a-06c9-429e-aae4-2940ff4d5a6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:22:46.096509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b5e0a-06c9-429e-aae4-2940ff4d5a6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:46.097935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b5e0a-06c9-429e-aae4-2940ff4d5a6c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:46.110049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 8/16] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:22:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:22:46.115317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43ef45-37be-4711-8720-3562ffcb3b0d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:46.115802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b5e0a-06c9-429e-aae4-2940ff4d5a6c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:46.143938 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:46.224947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b5e0a-06c9-429e-aae4-2940ff4d5a6c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:46.244118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:46.255627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d5b6f-89fd-4660-bbcf-0986b4e6a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:46.256308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d5b6f-89fd-4660-bbcf-0986b4e6a40a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:46.256595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d5b6f-89fd-4660-bbcf-0986b4e6a40a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:46.256874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d5b6f-89fd-4660-bbcf-0986b4e6a40a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:46.259549 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8daae-7129-4ea8-8aed-ab6e9d305900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:46.260230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8daae-7129-4ea8-8aed-ab6e9d305900 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:46.260516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8daae-7129-4ea8-8aed-ab6e9d305900 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:46.260842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8daae-7129-4ea8-8aed-ab6e9d305900 None admin] PATH_INFO: `/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:46.262360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8daae-7129-4ea8-8aed-ab6e9d305900 None admin] RBAC: Authorizing `identity:get_user(user_id=79d0d7df7a094b5ba4b9a9bb3b449967)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:46.264093 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8daae-7129-4ea8-8aed-ab6e9d305900 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:46.265684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 9/17] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:22:46 2022] GET /identity/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:22:46.267923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d5b6f-89fd-4660-bbcf-0986b4e6a40a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:46.269398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d5b6f-89fd-4660-bbcf-0986b4e6a40a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:46.271188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8daae-7129-4ea8-8aed-ab6e9d305900 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-613d5b6f-89fd-4660-bbcf-0986b4e6a40a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:22:46.288276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:22:46.289470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:22:46.290654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:22:46.290654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:22:46.290654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:22:46.290654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:22:46.290654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 9/18] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:22:46.297135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d5b6f-89fd-4660-bbcf-0986b4e6a40a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:46.362291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:46.368946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8daae-7129-4ea8-8aed-ab6e9d305900 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:46.396810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d5b6f-89fd-4660-bbcf-0986b4e6a40a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:46.402694 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69599e53-1ade-4cfd-99c9-2f531c214119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:46.403339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69599e53-1ade-4cfd-99c9-2f531c214119 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:46.403627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69599e53-1ade-4cfd-99c9-2f531c214119 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:46.403910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69599e53-1ade-4cfd-99c9-2f531c214119 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:46.405098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69599e53-1ade-4cfd-99c9-2f531c214119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:46.406562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69599e53-1ade-4cfd-99c9-2f531c214119 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:46.408163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 10/19] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:22:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:22:46.413966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69599e53-1ade-4cfd-99c9-2f531c214119 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:46.430132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650671c-f16c-49cb-a6f9-68fd56684397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:46.430739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650671c-f16c-49cb-a6f9-68fd56684397 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:46.431020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650671c-f16c-49cb-a6f9-68fd56684397 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:46.431294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650671c-f16c-49cb-a6f9-68fd56684397 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:46.431998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650671c-f16c-49cb-a6f9-68fd56684397 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:22:46.432307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650671c-f16c-49cb-a6f9-68fd56684397 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:46.433689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650671c-f16c-49cb-a6f9-68fd56684397 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:46.450603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 10/20] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:22:46.516689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69599e53-1ade-4cfd-99c9-2f531c214119 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:46.552008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2be327-9416-4956-8e5a-4ed20b6bc893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:46.552649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2be327-9416-4956-8e5a-4ed20b6bc893 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:46.552936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2be327-9416-4956-8e5a-4ed20b6bc893 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:46.553217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2be327-9416-4956-8e5a-4ed20b6bc893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:46.554029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2be327-9416-4956-8e5a-4ed20b6bc893 None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.domain.id=default, user.id=79d0d7df7a094b5ba4b9a9bb3b449967) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:22:46.554341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2be327-9416-4956-8e5a-4ed20b6bc893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:46.556392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2be327-9416-4956-8e5a-4ed20b6bc893 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:46.565731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 11/21] 10.222.0.64 () {56 vars in 1322 bytes} [Thu Aug 4 07:22:46 2022] GET /identity/v3/role_assignments?user.id=79d0d7df7a094b5ba4b9a9bb3b449967&scope.domain.id=default&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:22:48.662140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c10fa-a7b2-48c3-9d9a-744d3edf4374 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:48.662647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c10fa-a7b2-48c3-9d9a-744d3edf4374 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:48.662719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c10fa-a7b2-48c3-9d9a-744d3edf4374 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:48.664825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 11/22] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:48.676619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1267ab-0648-4a2b-a066-ea87899194fb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:48.677059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1267ab-0648-4a2b-a066-ea87899194fb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:48.677236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1267ab-0648-4a2b-a066-ea87899194fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:48.704988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7e1267ab-0648-4a2b-a066-ea87899194fb None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:48.706208 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7e1267ab-0648-4a2b-a066-ea87899194fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:48.731887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:48.790581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 12/23] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:22:48.825956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c10fa-a7b2-48c3-9d9a-744d3edf4374 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:48.865477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:48.958453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c10fa-a7b2-48c3-9d9a-744d3edf4374 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:49.006033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf14b7-7669-48a8-a985-6897f7d905ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:49.006683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf14b7-7669-48a8-a985-6897f7d905ca None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:49.006962 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf14b7-7669-48a8-a985-6897f7d905ca None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:49.007234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf14b7-7669-48a8-a985-6897f7d905ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:49.027718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf14b7-7669-48a8-a985-6897f7d905ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:49.029229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf14b7-7669-48a8-a985-6897f7d905ca None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4caf14b7-7669-48a8-a985-6897f7d905ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:22:49.051092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:22:49.051958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:22:49.052786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:22:49.052786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:22:49.052786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:22:49.052786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:22:49.052786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 12/24] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:22:49.061092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1267ab-0648-4a2b-a066-ea87899194fb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:49.204622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1267ab-0648-4a2b-a066-ea87899194fb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:49.254806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2ab08-1ab1-4040-9b17-7728dbf295a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:49.255391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ab08-1ab1-4040-9b17-7728dbf295a5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:49.255672 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ab08-1ab1-4040-9b17-7728dbf295a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:49.255949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ab08-1ab1-4040-9b17-7728dbf295a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:49.256671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2ab08-1ab1-4040-9b17-7728dbf295a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:22:49.256973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2ab08-1ab1-4040-9b17-7728dbf295a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:49.258410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2ab08-1ab1-4040-9b17-7728dbf295a5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:49.284136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 13/25] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:22:49.290876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf14b7-7669-48a8-a985-6897f7d905ca None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:49.325837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d7c10-58e9-4ef8-9b81-7b95bdff9d4e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:49.326459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d7c10-58e9-4ef8-9b81-7b95bdff9d4e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:49.326932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d7c10-58e9-4ef8-9b81-7b95bdff9d4e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:49.328847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 14/26] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:49.332864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98080f0d-b6ec-4fe9-bc72-f4d39df1721a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:49.333745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98080f0d-b6ec-4fe9-bc72-f4d39df1721a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:49.334221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98080f0d-b6ec-4fe9-bc72-f4d39df1721a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:49.336489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 15/27] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:49.337846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99f73b-0401-4140-9c17-69f6221e35b6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:49.338389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99f73b-0401-4140-9c17-69f6221e35b6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:49.338858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99f73b-0401-4140-9c17-69f6221e35b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:49.370543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6e99f73b-0401-4140-9c17-69f6221e35b6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:49.371791 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6e99f73b-0401-4140-9c17-69f6221e35b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:49.384333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:49.417591 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:49.434729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf14b7-7669-48a8-a985-6897f7d905ca None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:49.475555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a488c-da6e-4e7f-88db-47921a33f322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:49.477454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a488c-da6e-4e7f-88db-47921a33f322 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:49.478199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a488c-da6e-4e7f-88db-47921a33f322 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:49.478785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a488c-da6e-4e7f-88db-47921a33f322 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:49.479695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a488c-da6e-4e7f-88db-47921a33f322 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:49.481435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a488c-da6e-4e7f-88db-47921a33f322 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:49.557141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 16/28] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:22:49.558053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00164bf-5eda-47de-b72a-0f2f5a674ef6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:49.558381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00164bf-5eda-47de-b72a-0f2f5a674ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:49.558663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00164bf-5eda-47de-b72a-0f2f5a674ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:49.586505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c00164bf-5eda-47de-b72a-0f2f5a674ef6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:49.589866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c00164bf-5eda-47de-b72a-0f2f5a674ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:49.669185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 17/29] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:22:49.670541 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ff27d-6965-4810-aca2-4ac362f74037 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:49.670926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ff27d-6965-4810-aca2-4ac362f74037 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:49.671334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ff27d-6965-4810-aca2-4ac362f74037 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:49.672935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 18/30] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:49.677958 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ff27d-6965-4810-aca2-4ac362f74037 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:49.723006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 13/31] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:22:49 2022] POST /identity/v3/projects => generated 301 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:22:49.725596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a48f6a-cca5-4015-9cbf-63fe90d37833 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:49.726093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a48f6a-cca5-4015-9cbf-63fe90d37833 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:49.727261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a48f6a-cca5-4015-9cbf-63fe90d37833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:49.769471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-79a48f6a-cca5-4015-9cbf-63fe90d37833 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:49.776343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-79a48f6a-cca5-4015-9cbf-63fe90d37833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:49.826072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:49.861977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ff27d-6965-4810-aca2-4ac362f74037 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:49.902116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d695f28-a902-4448-bc9b-fe3ca04dd845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:49.903147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d695f28-a902-4448-bc9b-fe3ca04dd845 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:49.903774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d695f28-a902-4448-bc9b-fe3ca04dd845 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:49.904102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d695f28-a902-4448-bc9b-fe3ca04dd845 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:49.918215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d695f28-a902-4448-bc9b-fe3ca04dd845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:22:49.922004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 14/32] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:22:49.923031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a48f6a-cca5-4015-9cbf-63fe90d37833 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5d695f28-a902-4448-bc9b-fe3ca04dd845 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:22:49.923300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:22:49.924451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:22:49.925213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 19/33] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:22:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:22:49.931118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d695f28-a902-4448-bc9b-fe3ca04dd845 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:50.017695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:50.061685 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a48f6a-cca5-4015-9cbf-63fe90d37833 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:50.063084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d695f28-a902-4448-bc9b-fe3ca04dd845 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:50.100445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155c22e-c5b5-4061-a199-819f226e12fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:50.102054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155c22e-c5b5-4061-a199-819f226e12fd None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:50.102592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155c22e-c5b5-4061-a199-819f226e12fd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:50.103038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86608d82-d971-485c-9302-ee346d5ea5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:50.103231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155c22e-c5b5-4061-a199-819f226e12fd None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:50.103789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86608d82-d971-485c-9302-ee346d5ea5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:50.104180 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86608d82-d971-485c-9302-ee346d5ea5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:50.104529 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86608d82-d971-485c-9302-ee346d5ea5d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:50.104717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155c22e-c5b5-4061-a199-819f226e12fd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:50.105395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86608d82-d971-485c-9302-ee346d5ea5d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:22:50.105822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86608d82-d971-485c-9302-ee346d5ea5d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:50.106883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155c22e-c5b5-4061-a199-819f226e12fd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:50.107346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86608d82-d971-485c-9302-ee346d5ea5d5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:50.109897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:50.121074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 20/34] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:22:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:22:50.122142 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86608d82-d971-485c-9302-ee346d5ea5d5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:50.187122 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 15/35] 10.222.0.64 () {60 vars in 1131 bytes} [Thu Aug 4 07:22:49 2022] POST /identity/v3/roles => generated 225 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:22:50.188949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155c22e-c5b5-4061-a199-819f226e12fd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:50.277413 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86608d82-d971-485c-9302-ee346d5ea5d5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:50.324800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea8e42-9625-4591-b4cf-f4c7a0bbe175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:50.326028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea8e42-9625-4591-b4cf-f4c7a0bbe175 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:50.326547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea8e42-9625-4591-b4cf-f4c7a0bbe175 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:50.327020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea8e42-9625-4591-b4cf-f4c7a0bbe175 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:50.328113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea8e42-9625-4591-b4cf-f4c7a0bbe175 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:50.328994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155c22e-c5b5-4061-a199-819f226e12fd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:50.330219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea8e42-9625-4591-b4cf-f4c7a0bbe175 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:50.362565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:50.371334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6152e65-deb4-45f7-9d9e-646509421af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:50.372023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6152e65-deb4-45f7-9d9e-646509421af1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:50.372304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6152e65-deb4-45f7-9d9e-646509421af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:50.372590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6152e65-deb4-45f7-9d9e-646509421af1 None admin] PATH_INFO: `/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:50.373995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6152e65-deb4-45f7-9d9e-646509421af1 None admin] RBAC: Authorizing `identity:get_user(user_id=79d0d7df7a094b5ba4b9a9bb3b449967)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:50.375550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6152e65-deb4-45f7-9d9e-646509421af1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:50.377885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 16/36] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:22:50 2022] GET /identity/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967 => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:22:50.378618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1294f-c223-4546-8f5f-8e2d2da51b91 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:50.378905 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1294f-c223-4546-8f5f-8e2d2da51b91 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:50.379184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1294f-c223-4546-8f5f-8e2d2da51b91 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:50.380491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 17/37] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:50.386440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e50ad-7a58-416a-a42c-35ec4317d391 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:50.388633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e50ad-7a58-416a-a42c-35ec4317d391 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:50.388633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e50ad-7a58-416a-a42c-35ec4317d391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:50.409846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9d1e50ad-7a58-416a-a42c-35ec4317d391 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:50.411445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9d1e50ad-7a58-416a-a42c-35ec4317d391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:50.487645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 18/38] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:22:50.488842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e50ad-7a58-416a-a42c-35ec4317d391 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:50.508440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 21/39] 10.222.0.64 () {60 vars in 1131 bytes} [Thu Aug 4 07:22:50 2022] POST /identity/v3/roles => generated 219 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:22:50.511486 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea8e42-9625-4591-b4cf-f4c7a0bbe175 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:50.587332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:50.609793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e50ad-7a58-416a-a42c-35ec4317d391 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:50.633564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea8e42-9625-4591-b4cf-f4c7a0bbe175 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:50.641204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:50.651339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63db8d-59e7-4160-8556-d8290376ee81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:50.652034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63db8d-59e7-4160-8556-d8290376ee81 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:50.652331 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63db8d-59e7-4160-8556-d8290376ee81 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:50.659321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63db8d-59e7-4160-8556-d8290376ee81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:50.659321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63db8d-59e7-4160-8556-d8290376ee81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:50.659321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63db8d-59e7-4160-8556-d8290376ee81 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:50.659321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 19/40] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:22:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:22:50.659321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38be11-e23e-419f-b572-9e686ff0e895 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:50.659321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38be11-e23e-419f-b572-9e686ff0e895 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:50.659700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38be11-e23e-419f-b572-9e686ff0e895 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:50.659771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 20/41] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:50.664051 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38be11-e23e-419f-b572-9e686ff0e895 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:50.681880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a67b8-97fa-42e1-b1fb-e32f7733f696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:50.682521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a67b8-97fa-42e1-b1fb-e32f7733f696 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:50.682813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a67b8-97fa-42e1-b1fb-e32f7733f696 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:50.683085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a67b8-97fa-42e1-b1fb-e32f7733f696 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:50.683817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a67b8-97fa-42e1-b1fb-e32f7733f696 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:50.685397 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a67b8-97fa-42e1-b1fb-e32f7733f696 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:50.754098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 22/42] 10.222.0.64 () {60 vars in 1131 bytes} [Thu Aug 4 07:22:50 2022] POST /identity/v3/roles => generated 223 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:22:50.755478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e20f2d-6a4c-46d1-8e18-92b189af9d49 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:50.755776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e20f2d-6a4c-46d1-8e18-92b189af9d49 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:50.756086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e20f2d-6a4c-46d1-8e18-92b189af9d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:50.775957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-20e20f2d-6a4c-46d1-8e18-92b189af9d49 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:50.777599 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-20e20f2d-6a4c-46d1-8e18-92b189af9d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:50.782534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38be11-e23e-419f-b572-9e686ff0e895 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:50.831928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e05ead-ae5a-4d3c-815f-c0ee8a0043a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:50.832682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e05ead-ae5a-4d3c-815f-c0ee8a0043a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:50.832971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e05ead-ae5a-4d3c-815f-c0ee8a0043a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:50.833254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e05ead-ae5a-4d3c-815f-c0ee8a0043a9 None admin] PATH_INFO: `/v3/domains/default/users/79d0d7df7a094b5ba4b9a9bb3b449967/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:50.835405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e05ead-ae5a-4d3c-815f-c0ee8a0043a9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=79d0d7df7a094b5ba4b9a9bb3b449967, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:50.837053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e05ead-ae5a-4d3c-815f-c0ee8a0043a9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:50.854597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 23/43] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:22:50.879760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e20f2d-6a4c-46d1-8e18-92b189af9d49 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:50.914452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 21/44] 10.222.0.64 () {58 vars in 1361 bytes} [Thu Aug 4 07:22:50 2022] PUT /identity/v3/domains/default/users/79d0d7df7a094b5ba4b9a9bb3b449967/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:22:50.935339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:50.998170 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e20f2d-6a4c-46d1-8e18-92b189af9d49 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:51.032894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094b6ee-2e2c-43a9-8dc9-50c8b111fdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:51.033552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094b6ee-2e2c-43a9-8dc9-50c8b111fdae None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:51.033839 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094b6ee-2e2c-43a9-8dc9-50c8b111fdae None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:51.034119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094b6ee-2e2c-43a9-8dc9-50c8b111fdae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:51.035350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094b6ee-2e2c-43a9-8dc9-50c8b111fdae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:51.036855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094b6ee-2e2c-43a9-8dc9-50c8b111fdae None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:51.038482 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 24/45] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:22:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:22:51.045215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e05ead-ae5a-4d3c-815f-c0ee8a0043a9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:51.104963 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:51.147932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e05ead-ae5a-4d3c-815f-c0ee8a0043a9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:51.181695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db22e11-6d39-4763-8ad9-1e4aef2a0ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:51.182320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db22e11-6d39-4763-8ad9-1e4aef2a0ac4 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:51.182604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db22e11-6d39-4763-8ad9-1e4aef2a0ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:51.182879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db22e11-6d39-4763-8ad9-1e4aef2a0ac4 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:51.183561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db22e11-6d39-4763-8ad9-1e4aef2a0ac4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:51.185045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db22e11-6d39-4763-8ad9-1e4aef2a0ac4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:51.251971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 22/46] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:22:51 2022] POST /identity/v3/projects => generated 312 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:22:52.926000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79370ae1-f552-48e8-ae62-515716c8c30a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:52.926528 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79370ae1-f552-48e8-ae62-515716c8c30a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:52.926633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79370ae1-f552-48e8-ae62-515716c8c30a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:52.927924 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 25/47] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:52.933613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb91255-f0e9-43cc-aed5-db6cf0b74463 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:52.934020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb91255-f0e9-43cc-aed5-db6cf0b74463 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:52.934182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb91255-f0e9-43cc-aed5-db6cf0b74463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:52.952803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bbb91255-f0e9-43cc-aed5-db6cf0b74463 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:52.954000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bbb91255-f0e9-43cc-aed5-db6cf0b74463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:53.020466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 23/48] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:22:53.043310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79370ae1-f552-48e8-ae62-515716c8c30a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:53.142840 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79370ae1-f552-48e8-ae62-515716c8c30a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:53.158830 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:53.182264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb68e93-753b-417a-9ee4-0b75955d60d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:53.182264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb68e93-753b-417a-9ee4-0b75955d60d0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:53.182264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb68e93-753b-417a-9ee4-0b75955d60d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:53.182264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb68e93-753b-417a-9ee4-0b75955d60d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:53.188460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb68e93-753b-417a-9ee4-0b75955d60d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9eb68e93-753b-417a-9ee4-0b75955d60d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:22:53.195446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:22:53.196495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:22:53.197430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 26/49] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:22:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:22:53.202803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb91255-f0e9-43cc-aed5-db6cf0b74463 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:53.237433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:53.277338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e02f-72c3-4e04-89ec-4d77c2744b6f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:53.277625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e02f-72c3-4e04-89ec-4d77c2744b6f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:53.277901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e02f-72c3-4e04-89ec-4d77c2744b6f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:53.279242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 27/50] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:22:53.284144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82037d0e-6c6f-4aa0-b2af-818724dc10e9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:53.284435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82037d0e-6c6f-4aa0-b2af-818724dc10e9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:53.284710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82037d0e-6c6f-4aa0-b2af-818724dc10e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:53.302960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-82037d0e-6c6f-4aa0-b2af-818724dc10e9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:53.303139 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb91255-f0e9-43cc-aed5-db6cf0b74463 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:53.304823 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-82037d0e-6c6f-4aa0-b2af-818724dc10e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:53.337372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7520d3-7bc5-4986-b40e-79270a54476d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:53.338000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7520d3-7bc5-4986-b40e-79270a54476d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:53.338287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7520d3-7bc5-4986-b40e-79270a54476d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:53.338568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7520d3-7bc5-4986-b40e-79270a54476d None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:53.339286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7520d3-7bc5-4986-b40e-79270a54476d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:22:53.339598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7520d3-7bc5-4986-b40e-79270a54476d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:53.341029 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7520d3-7bc5-4986-b40e-79270a54476d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:53.352701 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 24/51] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:22:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:22:53.358133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7520d3-7bc5-4986-b40e-79270a54476d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:53.370800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 28/52] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:22:53.393325 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82037d0e-6c6f-4aa0-b2af-818724dc10e9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:53.455439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:53.459304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7520d3-7bc5-4986-b40e-79270a54476d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:53.493314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82037d0e-6c6f-4aa0-b2af-818724dc10e9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:53.493741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d934441-6e47-4e43-aae7-16eb04a32958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:53.494454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d934441-6e47-4e43-aae7-16eb04a32958 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:53.494810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d934441-6e47-4e43-aae7-16eb04a32958 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:53.495157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d934441-6e47-4e43-aae7-16eb04a32958 None admin] PATH_INFO: `/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:53.496430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d934441-6e47-4e43-aae7-16eb04a32958 None admin] RBAC: Authorizing `identity:get_user(user_id=79d0d7df7a094b5ba4b9a9bb3b449967)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:53.497875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d934441-6e47-4e43-aae7-16eb04a32958 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:53.499458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 25/53] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:22:53 2022] GET /identity/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:22:53.500490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:53.504930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d934441-6e47-4e43-aae7-16eb04a32958 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:53.528394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb0d25-7a12-4474-a737-1577c894bd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:53.529038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb0d25-7a12-4474-a737-1577c894bd8b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:53.529322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb0d25-7a12-4474-a737-1577c894bd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:53.529599 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb0d25-7a12-4474-a737-1577c894bd8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:53.530756 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb0d25-7a12-4474-a737-1577c894bd8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:53.532223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb0d25-7a12-4474-a737-1577c894bd8b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:53.533982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 29/54] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:22:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:22:53.540030 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb0d25-7a12-4474-a737-1577c894bd8b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:53.603901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d934441-6e47-4e43-aae7-16eb04a32958 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:53.634528 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb0d25-7a12-4474-a737-1577c894bd8b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:53.636887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a3076-07c2-43b1-afd1-bebbe260a8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:53.637528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a3076-07c2-43b1-afd1-bebbe260a8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:53.637817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a3076-07c2-43b1-afd1-bebbe260a8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:53.638096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a3076-07c2-43b1-afd1-bebbe260a8c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:53.639239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a3076-07c2-43b1-afd1-bebbe260a8c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:53.640726 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a3076-07c2-43b1-afd1-bebbe260a8c0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:53.642313 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 26/55] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:22:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:22:53.648468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a3076-07c2-43b1-afd1-bebbe260a8c0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:53.668890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09158d-8a26-4100-b94c-71b15766e1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:53.669513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09158d-8a26-4100-b94c-71b15766e1e6 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:53.669802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09158d-8a26-4100-b94c-71b15766e1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:53.670082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09158d-8a26-4100-b94c-71b15766e1e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:53.670771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09158d-8a26-4100-b94c-71b15766e1e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:53.672347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09158d-8a26-4100-b94c-71b15766e1e6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:53.682944 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:53.747604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a3076-07c2-43b1-afd1-bebbe260a8c0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:53.780900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dc61e-c6f6-4ad3-b6ab-54e4504fc4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:53.781520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dc61e-c6f6-4ad3-b6ab-54e4504fc4ee None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:53.781799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dc61e-c6f6-4ad3-b6ab-54e4504fc4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:53.782077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dc61e-c6f6-4ad3-b6ab-54e4504fc4ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:53.782859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2dc61e-c6f6-4ad3-b6ab-54e4504fc4ee None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.domain.id=default, user.id=79d0d7df7a094b5ba4b9a9bb3b449967) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:22:53.783064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 30/56] 10.222.0.64 () {60 vars in 1140 bytes} [Thu Aug 4 07:22:53 2022] POST /identity/v3/projects => generated 298 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:22:53.783185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2dc61e-c6f6-4ad3-b6ab-54e4504fc4ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:53.784855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2dc61e-c6f6-4ad3-b6ab-54e4504fc4ee None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:53.793536 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 27/57] 10.222.0.64 () {56 vars in 1322 bytes} [Thu Aug 4 07:22:53 2022] GET /identity/v3/role_assignments?user.id=79d0d7df7a094b5ba4b9a9bb3b449967&scope.domain.id=default&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:22:55.812572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fb16e-b300-4b4e-85e7-4dd172fe5098 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:55.813086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fb16e-b300-4b4e-85e7-4dd172fe5098 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:55.813169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fb16e-b300-4b4e-85e7-4dd172fe5098 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:55.814500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 31/58] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:55.819552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16eb8b-81ee-46ff-95cf-4164fb61f984 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:55.820577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16eb8b-81ee-46ff-95cf-4164fb61f984 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:55.820909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16eb8b-81ee-46ff-95cf-4164fb61f984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:55.839625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5c16eb8b-81ee-46ff-95cf-4164fb61f984 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:55.840850 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5c16eb8b-81ee-46ff-95cf-4164fb61f984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:55.904362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 28/59] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:22:55.928323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fb16e-b300-4b4e-85e7-4dd172fe5098 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:55.956833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:56.030790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fb16e-b300-4b4e-85e7-4dd172fe5098 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:56.063834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca78bf1-a5aa-4bc4-bf45-5e38975fd63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:56.064450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca78bf1-a5aa-4bc4-bf45-5e38975fd63c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:56.064739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca78bf1-a5aa-4bc4-bf45-5e38975fd63c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:56.065018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca78bf1-a5aa-4bc4-bf45-5e38975fd63c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:56.066172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca78bf1-a5aa-4bc4-bf45-5e38975fd63c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:56.067649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca78bf1-a5aa-4bc4-bf45-5e38975fd63c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:56.069272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 32/60] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:22:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:22:56.074925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16eb8b-81ee-46ff-95cf-4164fb61f984 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:56.101601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:56.170525 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16eb8b-81ee-46ff-95cf-4164fb61f984 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:56.203059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6382718-f526-4457-8ec6-69f798efef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:56.203675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6382718-f526-4457-8ec6-69f798efef4f None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:56.203961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6382718-f526-4457-8ec6-69f798efef4f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:56.204272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6382718-f526-4457-8ec6-69f798efef4f None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:56.204955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6382718-f526-4457-8ec6-69f798efef4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:22:56.206392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6382718-f526-4457-8ec6-69f798efef4f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:22:56.311945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 29/61] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:22:56 2022] POST /identity/v3/users => generated 275 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:22:59.427167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ab7fd-4593-4202-a78a-4f13afdc8927 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:59.428230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ab7fd-4593-4202-a78a-4f13afdc8927 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:59.428767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ab7fd-4593-4202-a78a-4f13afdc8927 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:59.431132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 33/62] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:59.436257 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2e547-3a2a-4cef-bf85-a07c7bc9d115 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:59.437088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2e547-3a2a-4cef-bf85-a07c7bc9d115 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:59.437583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2e547-3a2a-4cef-bf85-a07c7bc9d115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:59.443877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d7dac-9cc6-43b2-9952-45c8428b59db None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:59.444552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d7dac-9cc6-43b2-9952-45c8428b59db None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:59.445561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d7dac-9cc6-43b2-9952-45c8428b59db None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:59.447104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 34/63] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:22:59.452407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f62482-8969-4ceb-a6c3-5cc7dd48fa45 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:59.452965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f62482-8969-4ceb-a6c3-5cc7dd48fa45 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:59.453448 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f62482-8969-4ceb-a6c3-5cc7dd48fa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:59.458144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d0b2e547-3a2a-4cef-bf85-a07c7bc9d115 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:59.459336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d0b2e547-3a2a-4cef-bf85-a07c7bc9d115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:59.493126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-19f62482-8969-4ceb-a6c3-5cc7dd48fa45 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:59.494371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-19f62482-8969-4ceb-a6c3-5cc7dd48fa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:59.579551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 30/64] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:22:59.607383 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 35/65] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:22:59.611697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2e547-3a2a-4cef-bf85-a07c7bc9d115 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:59.639343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:59.641025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf9a0-9908-4960-a726-8720d4a3755c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:59.641320 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf9a0-9908-4960-a726-8720d4a3755c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:59.641603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf9a0-9908-4960-a726-8720d4a3755c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:59.643391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 36/66] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:22:59.648669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb6a33-4dc0-41cd-9504-65c107d88968 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:59.649150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb6a33-4dc0-41cd-9504-65c107d88968 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:59.649502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb6a33-4dc0-41cd-9504-65c107d88968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:59.672561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e9cb6a33-4dc0-41cd-9504-65c107d88968 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:22:59.673707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e9cb6a33-4dc0-41cd-9504-65c107d88968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:22:59.698346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:59.749666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2e547-3a2a-4cef-bf85-a07c7bc9d115 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:59.767188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 37/67] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:22:59.768509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb6a33-4dc0-41cd-9504-65c107d88968 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:59.810603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01274b-3f7b-4e6f-ad6c-b60a7b6cff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:59.811255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01274b-3f7b-4e6f-ad6c-b60a7b6cff4d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:59.811537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01274b-3f7b-4e6f-ad6c-b60a7b6cff4d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:59.811813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01274b-3f7b-4e6f-ad6c-b60a7b6cff4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:59.823631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01274b-3f7b-4e6f-ad6c-b60a7b6cff4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9b01274b-3f7b-4e6f-ad6c-b60a7b6cff4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:22:59.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:22:59.831140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 31/68] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:22:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:22:59.832436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01274b-3f7b-4e6f-ad6c-b60a7b6cff4d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:59.899063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb6a33-4dc0-41cd-9504-65c107d88968 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:59.912529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:22:59.940583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01274b-3f7b-4e6f-ad6c-b60a7b6cff4d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:22:59.947796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfbe48-90a6-45ce-a0b5-a59342282e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:59.948829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfbe48-90a6-45ce-a0b5-a59342282e2f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:59.949330 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfbe48-90a6-45ce-a0b5-a59342282e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:59.949833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfbe48-90a6-45ce-a0b5-a59342282e2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:22:59.964229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfbe48-90a6-45ce-a0b5-a59342282e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-00dfbe48-90a6-45ce-a0b5-a59342282e2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:22:59.972623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:22:59.973465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:22:59.974229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:22:59.977618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 38/69] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:22:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:22:59.978867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfbe48-90a6-45ce-a0b5-a59342282e2f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:22:59.987438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c36543-02d9-4c68-96b9-73b183941c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:22:59.988118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c36543-02d9-4c68-96b9-73b183941c71 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:22:59.988563 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c36543-02d9-4c68-96b9-73b183941c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:22:59.989027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c36543-02d9-4c68-96b9-73b183941c71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.000643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c36543-02d9-4c68-96b9-73b183941c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:00.001663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e4c36543-02d9-4c68-96b9-73b183941c71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:00.006030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:00.007229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 32/70] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:22:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:00.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c36543-02d9-4c68-96b9-73b183941c71 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:00.096131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfbe48-90a6-45ce-a0b5-a59342282e2f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:00.127350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c36543-02d9-4c68-96b9-73b183941c71 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:00.132190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1660f-822b-4028-83e6-67993cd27e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:00.132825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1660f-822b-4028-83e6-67993cd27e52 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.133114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1660f-822b-4028-83e6-67993cd27e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.133392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1660f-822b-4028-83e6-67993cd27e52 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.134114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1660f-822b-4028-83e6-67993cd27e52 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:00.134422 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1660f-822b-4028-83e6-67993cd27e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:00.135814 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1660f-822b-4028-83e6-67993cd27e52 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:00.144465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 39/71] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:22:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:00.145165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1660f-822b-4028-83e6-67993cd27e52 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:00.162903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d10f8-d770-42a8-86e7-bc878b1ee38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:00.164111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d10f8-d770-42a8-86e7-bc878b1ee38e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.164399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d10f8-d770-42a8-86e7-bc878b1ee38e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.164676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d10f8-d770-42a8-86e7-bc878b1ee38e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.165396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d10f8-d770-42a8-86e7-bc878b1ee38e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:00.165717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d10f8-d770-42a8-86e7-bc878b1ee38e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:00.167108 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d10f8-d770-42a8-86e7-bc878b1ee38e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:00.168539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:00.176381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 33/72] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:00.177652 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7471dc3-c35c-4de2-aa4f-6f95414cce69 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.177942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7471dc3-c35c-4de2-aa4f-6f95414cce69 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.178227 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7471dc3-c35c-4de2-aa4f-6f95414cce69 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.179682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 34/73] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:00.180474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7471dc3-c35c-4de2-aa4f-6f95414cce69 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:00.227463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:00.249020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1660f-822b-4028-83e6-67993cd27e52 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:00.282572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6590deb-e67c-4589-90d6-0adc267d73d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:00.283198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590deb-e67c-4589-90d6-0adc267d73d6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.283490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590deb-e67c-4589-90d6-0adc267d73d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.283793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590deb-e67c-4589-90d6-0adc267d73d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.284008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7471dc3-c35c-4de2-aa4f-6f95414cce69 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:00.284529 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6590deb-e67c-4589-90d6-0adc267d73d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:00.284842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6590deb-e67c-4589-90d6-0adc267d73d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:00.286230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6590deb-e67c-4589-90d6-0adc267d73d6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:00.294599 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 40/74] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:00.295845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ce9fc-757f-4c1d-87fe-f22c5cac07e5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.296199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ce9fc-757f-4c1d-87fe-f22c5cac07e5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.296490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ce9fc-757f-4c1d-87fe-f22c5cac07e5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.297715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 41/75] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:00.301683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ce9fc-757f-4c1d-87fe-f22c5cac07e5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:00.322216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c765dd9-7ce3-41c2-900b-58d53dbbeb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:00.322933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c765dd9-7ce3-41c2-900b-58d53dbbeb44 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.323225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c765dd9-7ce3-41c2-900b-58d53dbbeb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.323502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c765dd9-7ce3-41c2-900b-58d53dbbeb44 None admin] PATH_INFO: `/v3/users/60b0d29f45f6410f8f4e1856e03bc22b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.337219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c765dd9-7ce3-41c2-900b-58d53dbbeb44 None admin] RBAC: Authorizing `identity:get_user(user_id=60b0d29f45f6410f8f4e1856e03bc22b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:00.338675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c765dd9-7ce3-41c2-900b-58d53dbbeb44 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:00.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 35/76] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:00 2022] GET /identity/v3/users/60b0d29f45f6410f8f4e1856e03bc22b => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:00.343032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296d0ab-d6b2-4c6b-b77d-568f0b590466 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.343340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296d0ab-d6b2-4c6b-b77d-568f0b590466 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.343780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296d0ab-d6b2-4c6b-b77d-568f0b590466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.361031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7296d0ab-d6b2-4c6b-b77d-568f0b590466 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:00.362222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7296d0ab-d6b2-4c6b-b77d-568f0b590466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:00.399504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ce9fc-757f-4c1d-87fe-f22c5cac07e5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:00.411093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:00.428688 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 36/77] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:00.429994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296d0ab-d6b2-4c6b-b77d-568f0b590466 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:00.433529 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874527a-8e49-41d4-b052-30be2c1927f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:00.434176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874527a-8e49-41d4-b052-30be2c1927f8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.434461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874527a-8e49-41d4-b052-30be2c1927f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.434741 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874527a-8e49-41d4-b052-30be2c1927f8 None admin] PATH_INFO: `/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.435918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874527a-8e49-41d4-b052-30be2c1927f8 None admin] RBAC: Authorizing `identity:get_user(user_id=79d0d7df7a094b5ba4b9a9bb3b449967)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:00.437387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874527a-8e49-41d4-b052-30be2c1927f8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:00.438989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 42/78] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:00 2022] GET /identity/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:00.440159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afbeb8-6a50-492b-af2b-80eccbdb81ca None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.440398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afbeb8-6a50-492b-af2b-80eccbdb81ca None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.440679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afbeb8-6a50-492b-af2b-80eccbdb81ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.459861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:00.466114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e6afbeb8-6a50-492b-af2b-80eccbdb81ca None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:00.470398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e6afbeb8-6a50-492b-af2b-80eccbdb81ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:00.531741 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 43/79] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:00.532847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afbeb8-6a50-492b-af2b-80eccbdb81ca None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:00.538985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296d0ab-d6b2-4c6b-b77d-568f0b590466 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:00.575651 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e8a1b-45c5-45a8-9da8-50f4760f99df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:00.576338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e8a1b-45c5-45a8-9da8-50f4760f99df None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.576625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e8a1b-45c5-45a8-9da8-50f4760f99df None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.576907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e8a1b-45c5-45a8-9da8-50f4760f99df None admin] PATH_INFO: `/v3/users/60b0d29f45f6410f8f4e1856e03bc22b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.578090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e8a1b-45c5-45a8-9da8-50f4760f99df None admin] RBAC: Authorizing `identity:get_user(user_id=60b0d29f45f6410f8f4e1856e03bc22b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:00.579554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e8a1b-45c5-45a8-9da8-50f4760f99df None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:00.581176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 37/80] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:00 2022] GET /identity/v3/users/60b0d29f45f6410f8f4e1856e03bc22b => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:00.584881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e8a1b-45c5-45a8-9da8-50f4760f99df None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:00.632660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afbeb8-6a50-492b-af2b-80eccbdb81ca None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:00.636390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:00.666439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058cfd8-3aff-4942-9a5a-857eaee16a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:00.667174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058cfd8-3aff-4942-9a5a-857eaee16a97 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.667474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058cfd8-3aff-4942-9a5a-857eaee16a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.667752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058cfd8-3aff-4942-9a5a-857eaee16a97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.671575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:00.683357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e8a1b-45c5-45a8-9da8-50f4760f99df None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:00.716347 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752f7db-969e-4cd8-bbfd-44b671611287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:00.717075 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752f7db-969e-4cd8-bbfd-44b671611287 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.717361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752f7db-969e-4cd8-bbfd-44b671611287 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.717642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752f7db-969e-4cd8-bbfd-44b671611287 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.735957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058cfd8-3aff-4942-9a5a-857eaee16a97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:00.737463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058cfd8-3aff-4942-9a5a-857eaee16a97 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:00.785494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752f7db-969e-4cd8-bbfd-44b671611287 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:00.786886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752f7db-969e-4cd8-bbfd-44b671611287 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:00.834803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 44/81] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:00.840336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:00.841012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058cfd8-3aff-4942-9a5a-857eaee16a97 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:00.874276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:00.887154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 38/82] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:00.888089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752f7db-969e-4cd8-bbfd-44b671611287 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:00.943108 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058cfd8-3aff-4942-9a5a-857eaee16a97 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:00.976918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b94d0-42f0-424c-92d4-7425e64e1e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:00.977563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b94d0-42f0-424c-92d4-7425e64e1e84 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:00.977854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b94d0-42f0-424c-92d4-7425e64e1e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:00.978135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b94d0-42f0-424c-92d4-7425e64e1e84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:00.979287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b94d0-42f0-424c-92d4-7425e64e1e84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:00.980764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b94d0-42f0-424c-92d4-7425e64e1e84 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:00.982503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 45/83] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:00.983185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b94d0-42f0-424c-92d4-7425e64e1e84 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:00.994673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752f7db-969e-4cd8-bbfd-44b671611287 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:01.027746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cd44f-1952-4bb5-8797-958ce2e78956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:01.028416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cd44f-1952-4bb5-8797-958ce2e78956 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:01.028704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cd44f-1952-4bb5-8797-958ce2e78956 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:01.028982 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cd44f-1952-4bb5-8797-958ce2e78956 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:01.039057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cd44f-1952-4bb5-8797-958ce2e78956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6e5cd44f-1952-4bb5-8797-958ce2e78956 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:01.044438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:01.045364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:01.046111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 39/84] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:01.048786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cd44f-1952-4bb5-8797-958ce2e78956 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:01.063039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:01.080350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b94d0-42f0-424c-92d4-7425e64e1e84 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:01.082765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:01.113910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b6944-24ab-4e84-962a-f7c5748bd95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:01.114524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b6944-24ab-4e84-962a-f7c5748bd95d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:01.114808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b6944-24ab-4e84-962a-f7c5748bd95d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:01.115084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b6944-24ab-4e84-962a-f7c5748bd95d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:01.148399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cd44f-1952-4bb5-8797-958ce2e78956 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:01.182188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b6944-24ab-4e84-962a-f7c5748bd95d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:01.182461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c7b47-89af-4708-aca4-79bbed37319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:01.183102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c7b47-89af-4708-aca4-79bbed37319a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:01.183390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c7b47-89af-4708-aca4-79bbed37319a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:01.183662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b6944-24ab-4e84-962a-f7c5748bd95d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:01.183775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c7b47-89af-4708-aca4-79bbed37319a None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:01.184853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c7b47-89af-4708-aca4-79bbed37319a None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:01.186295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c7b47-89af-4708-aca4-79bbed37319a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:01.187953 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 40/85] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:01 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:01.188735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c7b47-89af-4708-aca4-79bbed37319a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:01.270920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:01.281998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 46/86] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:01.282824 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b6944-24ab-4e84-962a-f7c5748bd95d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:01.285930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:01.290146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c7b47-89af-4708-aca4-79bbed37319a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:01.324414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d6513-faf5-4bd2-aa11-41889f1ffd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:01.325050 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d6513-faf5-4bd2-aa11-41889f1ffd69 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:01.325339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d6513-faf5-4bd2-aa11-41889f1ffd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:01.325622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d6513-faf5-4bd2-aa11-41889f1ffd69 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:01.326789 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d6513-faf5-4bd2-aa11-41889f1ffd69 None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:01.328272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d6513-faf5-4bd2-aa11-41889f1ffd69 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:01.329916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 41/87] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:01 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:01.330672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d6513-faf5-4bd2-aa11-41889f1ffd69 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:01.380387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b6944-24ab-4e84-962a-f7c5748bd95d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:01.414122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be5d5a-39d8-4f42-98b3-606247deb6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:01.414744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be5d5a-39d8-4f42-98b3-606247deb6e9 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:01.415030 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be5d5a-39d8-4f42-98b3-606247deb6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:01.415312 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be5d5a-39d8-4f42-98b3-606247deb6e9 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:01.416028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be5d5a-39d8-4f42-98b3-606247deb6e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:01.417488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be5d5a-39d8-4f42-98b3-606247deb6e9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:01.428949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d6513-faf5-4bd2-aa11-41889f1ffd69 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:01.462400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305194a-c8b3-46e2-ab2a-539d152768a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:01.463013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305194a-c8b3-46e2-ab2a-539d152768a9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:01.463299 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305194a-c8b3-46e2-ab2a-539d152768a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:01.463578 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305194a-c8b3-46e2-ab2a-539d152768a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:01.465817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305194a-c8b3-46e2-ab2a-539d152768a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:01.465817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305194a-c8b3-46e2-ab2a-539d152768a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:01.466041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305194a-c8b3-46e2-ab2a-539d152768a9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:01.473709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 42/88] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:01.475612 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305194a-c8b3-46e2-ab2a-539d152768a9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:01.487876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:01.519268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 47/89] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:23:01 2022] POST /identity/v3/users => generated 289 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:23:01.520133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be5d5a-39d8-4f42-98b3-606247deb6e9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:01.585166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305194a-c8b3-46e2-ab2a-539d152768a9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:01.590300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:01.624497 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4faed17-6353-475d-bd16-fbf1d5474b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:01.625256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faed17-6353-475d-bd16-fbf1d5474b47 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:01.625548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faed17-6353-475d-bd16-fbf1d5474b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:01.625828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faed17-6353-475d-bd16-fbf1d5474b47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:01.626638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4faed17-6353-475d-bd16-fbf1d5474b47 None admin] RBAC: Adding query filter params (role.id=525e499f44594268b4f362c62bd35e00, scope.project.id=ce53e46475f341a2af8bf9da4700fffe, user.id=60b0d29f45f6410f8f4e1856e03bc22b) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:01.626958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4faed17-6353-475d-bd16-fbf1d5474b47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:01.628463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4faed17-6353-475d-bd16-fbf1d5474b47 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:01.634995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 43/90] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:01 2022] GET /identity/v3/role_assignments?user.id=60b0d29f45f6410f8f4e1856e03bc22b&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=525e499f44594268b4f362c62bd35e00 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:01.635969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be5d5a-39d8-4f42-98b3-606247deb6e9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:01.656013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4faed17-6353-475d-bd16-fbf1d5474b47 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:01.695462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e8afd-86ba-4f7d-8c0e-bd097fdf98ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:01.696672 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e8afd-86ba-4f7d-8c0e-bd097fdf98ae None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:01.697256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e8afd-86ba-4f7d-8c0e-bd097fdf98ae None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:01.697547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e8afd-86ba-4f7d-8c0e-bd097fdf98ae None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:01.700860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e8afd-86ba-4f7d-8c0e-bd097fdf98ae None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:01.702392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e8afd-86ba-4f7d-8c0e-bd097fdf98ae None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:01.705404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 48/91] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:01 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:01.706273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e8afd-86ba-4f7d-8c0e-bd097fdf98ae None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:01.760871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:01.794166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4faed17-6353-475d-bd16-fbf1d5474b47 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:01.840951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56024e-3c88-45ca-a548-bceb45d56be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:01.842007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56024e-3c88-45ca-a548-bceb45d56be0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:01.842469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56024e-3c88-45ca-a548-bceb45d56be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:01.842985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56024e-3c88-45ca-a548-bceb45d56be0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:01.843874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56024e-3c88-45ca-a548-bceb45d56be0 None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=ce53e46475f341a2af8bf9da4700fffe, user.id=79d0d7df7a094b5ba4b9a9bb3b449967) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:01.844662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56024e-3c88-45ca-a548-bceb45d56be0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:01.846334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56024e-3c88-45ca-a548-bceb45d56be0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:01.853182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 44/92] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:01 2022] GET /identity/v3/role_assignments?user.id=79d0d7df7a094b5ba4b9a9bb3b449967&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:01.854130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56024e-3c88-45ca-a548-bceb45d56be0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:01.857469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e8afd-86ba-4f7d-8c0e-bd097fdf98ae None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:01.899195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:01.911751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2a5f1-5345-4a6f-b59d-9cea4e0e5289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:01.912936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2a5f1-5345-4a6f-b59d-9cea4e0e5289 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:01.914371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2a5f1-5345-4a6f-b59d-9cea4e0e5289 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:01.914855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2a5f1-5345-4a6f-b59d-9cea4e0e5289 None admin] PATH_INFO: `/v3/users/60b0d29f45f6410f8f4e1856e03bc22b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:01.916886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2a5f1-5345-4a6f-b59d-9cea4e0e5289 None admin] RBAC: Authorizing `identity:get_user(user_id=60b0d29f45f6410f8f4e1856e03bc22b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:01.919453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2a5f1-5345-4a6f-b59d-9cea4e0e5289 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:01.922498 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 49/93] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:01 2022] GET /identity/v3/users/60b0d29f45f6410f8f4e1856e03bc22b => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:01.928356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2a5f1-5345-4a6f-b59d-9cea4e0e5289 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:02.018770 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56024e-3c88-45ca-a548-bceb45d56be0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:02.091275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2a5f1-5345-4a6f-b59d-9cea4e0e5289 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:02.094175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2576614-09e1-450b-bdbd-28357222d24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:02.094915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576614-09e1-450b-bdbd-28357222d24b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:02.095206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576614-09e1-450b-bdbd-28357222d24b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:02.095493 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576614-09e1-450b-bdbd-28357222d24b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:02.096337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2576614-09e1-450b-bdbd-28357222d24b None admin] RBAC: Adding query filter params (role.id=e3c5a858a45c41db8ff3e883788d75bc, scope.project.id=ce53e46475f341a2af8bf9da4700fffe, user.id=60b0d29f45f6410f8f4e1856e03bc22b) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:02.096657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2576614-09e1-450b-bdbd-28357222d24b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:02.098132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2576614-09e1-450b-bdbd-28357222d24b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:02.110376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 45/94] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:01 2022] GET /identity/v3/role_assignments?user.id=60b0d29f45f6410f8f4e1856e03bc22b&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=e3c5a858a45c41db8ff3e883788d75bc => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:02.132908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265d78c-6b46-4065-8923-f699c88a87c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:02.133955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265d78c-6b46-4065-8923-f699c88a87c2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:02.134497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265d78c-6b46-4065-8923-f699c88a87c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:02.135013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265d78c-6b46-4065-8923-f699c88a87c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:02.219112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:02.231685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265d78c-6b46-4065-8923-f699c88a87c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:02.233373 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265d78c-6b46-4065-8923-f699c88a87c2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:02.412884 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 50/95] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:02.418571 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2576614-09e1-450b-bdbd-28357222d24b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:02.426766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:02.556812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2576614-09e1-450b-bdbd-28357222d24b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:02.608830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bd294-3040-478b-8fb4-cd37d9621f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:02.609580 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bd294-3040-478b-8fb4-cd37d9621f9c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:02.609762 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bd294-3040-478b-8fb4-cd37d9621f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:02.610043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bd294-3040-478b-8fb4-cd37d9621f9c None admin] PATH_INFO: `/v3/projects/6c34cca62b0d4a439cd929a3b9f9de41` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:02.617286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77bd294-3040-478b-8fb4-cd37d9621f9c None admin] RBAC: Authorizing `identity:get_project(project_id=6c34cca62b0d4a439cd929a3b9f9de41)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:02.618767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77bd294-3040-478b-8fb4-cd37d9621f9c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:02.620508 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 46/96] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:02 2022] GET /identity/v3/projects/6c34cca62b0d4a439cd929a3b9f9de41 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:02.626526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265d78c-6b46-4065-8923-f699c88a87c2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:02.758782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:02.764409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265d78c-6b46-4065-8923-f699c88a87c2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:02.811355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6074081-b5c3-4014-b894-4067fa047aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:02.811999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6074081-b5c3-4014-b894-4067fa047aba None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:02.812295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6074081-b5c3-4014-b894-4067fa047aba None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:02.812574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6074081-b5c3-4014-b894-4067fa047aba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:02.813375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6074081-b5c3-4014-b894-4067fa047aba None admin] RBAC: Adding query filter params (role.id=525e499f44594268b4f362c62bd35e00, scope.project.id=6c34cca62b0d4a439cd929a3b9f9de41, user.id=60b0d29f45f6410f8f4e1856e03bc22b) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:02.813688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6074081-b5c3-4014-b894-4067fa047aba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:02.815129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6074081-b5c3-4014-b894-4067fa047aba None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:02.823999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 51/97] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:02 2022] GET /identity/v3/role_assignments?user.id=60b0d29f45f6410f8f4e1856e03bc22b&scope.project.id=6c34cca62b0d4a439cd929a3b9f9de41&role.id=525e499f44594268b4f362c62bd35e00 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:04.417807 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d12d34-2dcb-4a7c-9926-f831c28025ef None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:04.418281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d12d34-2dcb-4a7c-9926-f831c28025ef None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:04.418281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d12d34-2dcb-4a7c-9926-f831c28025ef None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:04.419627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 47/98] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:04.427018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c3668-57b0-436e-97d5-353611090992 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:04.427429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c3668-57b0-436e-97d5-353611090992 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:04.427615 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c3668-57b0-436e-97d5-353611090992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:04.450212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-544c3668-57b0-436e-97d5-353611090992 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:04.451420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-544c3668-57b0-436e-97d5-353611090992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:04.550254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 52/99] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:04.584666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d12d34-2dcb-4a7c-9926-f831c28025ef None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:04.673333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1ae07-a1ad-4675-8daa-c889837c1c30 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:04.673632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1ae07-a1ad-4675-8daa-c889837c1c30 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:04.673896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1ae07-a1ad-4675-8daa-c889837c1c30 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:04.675425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 53/100] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:04.681084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7c751-4b99-4d29-ad86-c4b16da4b1ae None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:04.681370 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7c751-4b99-4d29-ad86-c4b16da4b1ae None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:04.681654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7c751-4b99-4d29-ad86-c4b16da4b1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:04.704398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:04.719453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b6b7c751-4b99-4d29-ad86-c4b16da4b1ae None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:04.720731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b6b7c751-4b99-4d29-ad86-c4b16da4b1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:04.784677 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d12d34-2dcb-4a7c-9926-f831c28025ef None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:04.842403 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f527c-51d5-4392-9e63-e49f3765b067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:04.843058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f527c-51d5-4392-9e63-e49f3765b067 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:04.843342 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f527c-51d5-4392-9e63-e49f3765b067 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:04.843622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f527c-51d5-4392-9e63-e49f3765b067 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:04.859261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f527c-51d5-4392-9e63-e49f3765b067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6c6f527c-51d5-4392-9e63-e49f3765b067 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:04.865232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:04.866625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:04.867400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 48/101] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:04.873871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f527c-51d5-4392-9e63-e49f3765b067 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:04.902376 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 54/102] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:04.945693 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7c751-4b99-4d29-ad86-c4b16da4b1ae None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:04.980371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:05.024581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f527c-51d5-4392-9e63-e49f3765b067 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:05.053179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:05.062219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d38e5-3e5a-48dd-b7b6-458b23f4d7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:05.062864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d38e5-3e5a-48dd-b7b6-458b23f4d7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.063148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d38e5-3e5a-48dd-b7b6-458b23f4d7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.063424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d38e5-3e5a-48dd-b7b6-458b23f4d7a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.064203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d38e5-3e5a-48dd-b7b6-458b23f4d7a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:05.064513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d38e5-3e5a-48dd-b7b6-458b23f4d7a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:05.066560 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d38e5-3e5a-48dd-b7b6-458b23f4d7a9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:05.082396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 49/103] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:05.082396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cf672-ea68-4e16-b879-c889d4e33e71 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.082396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cf672-ea68-4e16-b879-c889d4e33e71 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.082396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cf672-ea68-4e16-b879-c889d4e33e71 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.082396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 50/104] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:05.083245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32de9b-5c85-4869-a436-8783dea3fefb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.083520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32de9b-5c85-4869-a436-8783dea3fefb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.083842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32de9b-5c85-4869-a436-8783dea3fefb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.085744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 51/105] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:05.087055 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367942b-609e-4cda-9862-93c6563161d6 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.087335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367942b-609e-4cda-9862-93c6563161d6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.087650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367942b-609e-4cda-9862-93c6563161d6 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.093489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 52/106] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:05.094557 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367942b-609e-4cda-9862-93c6563161d6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:05.102478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7c751-4b99-4d29-ad86-c4b16da4b1ae None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:05.138359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b3d4e-586c-4e39-bcb5-4e42cc4595d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:05.139007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3d4e-586c-4e39-bcb5-4e42cc4595d7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.139299 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3d4e-586c-4e39-bcb5-4e42cc4595d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.139578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b3d4e-586c-4e39-bcb5-4e42cc4595d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.140756 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b3d4e-586c-4e39-bcb5-4e42cc4595d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:05.142206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b3d4e-586c-4e39-bcb5-4e42cc4595d7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:05.143864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 55/107] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:05.146188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacea30-4273-441b-a418-bc756deded41 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.147159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacea30-4273-441b-a418-bc756deded41 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.148380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacea30-4273-441b-a418-bc756deded41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.166291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4dacea30-4273-441b-a418-bc756deded41 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:05.167551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4dacea30-4273-441b-a418-bc756deded41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:05.210455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367942b-609e-4cda-9862-93c6563161d6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:05.248011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 56/108] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:05.248011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e36564-1888-4042-82a5-572f6874d235 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.248011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e36564-1888-4042-82a5-572f6874d235 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.248011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e36564-1888-4042-82a5-572f6874d235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.256335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bc759-b4ff-48d4-868d-292834fb5b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:05.257643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bc759-b4ff-48d4-868d-292834fb5b0a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.257995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bc759-b4ff-48d4-868d-292834fb5b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.258344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bc759-b4ff-48d4-868d-292834fb5b0a None admin] PATH_INFO: `/v3/users/60b0d29f45f6410f8f4e1856e03bc22b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.259677 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bc759-b4ff-48d4-868d-292834fb5b0a None admin] RBAC: Authorizing `identity:get_user(user_id=60b0d29f45f6410f8f4e1856e03bc22b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:05.261353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bc759-b4ff-48d4-868d-292834fb5b0a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:05.263204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 53/109] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:05 2022] GET /identity/v3/users/60b0d29f45f6410f8f4e1856e03bc22b => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:05.264399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c466b1-75b0-4b46-aacd-67df67580788 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.264688 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c466b1-75b0-4b46-aacd-67df67580788 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.265012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c466b1-75b0-4b46-aacd-67df67580788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.270577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-01e36564-1888-4042-82a5-572f6874d235 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:05.271838 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-01e36564-1888-4042-82a5-572f6874d235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:05.284300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-65c466b1-75b0-4b46-aacd-67df67580788 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:05.285551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-65c466b1-75b0-4b46-aacd-67df67580788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:05.286751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:05.344501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 57/110] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:05.345625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e36564-1888-4042-82a5-572f6874d235 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:05.355822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:05.372032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 54/111] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:05.373155 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c466b1-75b0-4b46-aacd-67df67580788 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:05.466009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e36564-1888-4042-82a5-572f6874d235 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:05.480385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c466b1-75b0-4b46-aacd-67df67580788 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:05.502197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe16093-aabf-4017-b692-d82217584ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:05.503084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe16093-aabf-4017-b692-d82217584ece None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.503417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe16093-aabf-4017-b692-d82217584ece None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.503849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe16093-aabf-4017-b692-d82217584ece None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.504879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe16093-aabf-4017-b692-d82217584ece None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:05.506315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe16093-aabf-4017-b692-d82217584ece None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:05.515252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcda408-6bc2-4136-8704-ef539fb8b961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:05.515911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda408-6bc2-4136-8704-ef539fb8b961 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.516229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda408-6bc2-4136-8704-ef539fb8b961 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.516513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda408-6bc2-4136-8704-ef539fb8b961 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.517682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcda408-6bc2-4136-8704-ef539fb8b961 None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:05.519137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcda408-6bc2-4136-8704-ef539fb8b961 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:05.520866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 55/112] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:05 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:05.521653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcda408-6bc2-4136-8704-ef539fb8b961 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:05.581641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:05.582889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:05.589516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 58/113] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:23:05 2022] POST /identity/v3/projects => generated 302 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:23:05.590163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe16093-aabf-4017-b692-d82217584ece None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:05.636700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcda408-6bc2-4136-8704-ef539fb8b961 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:05.678094 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e9fdf-600f-447f-b250-5b504f217709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:05.678771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e9fdf-600f-447f-b250-5b504f217709 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.679123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e9fdf-600f-447f-b250-5b504f217709 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.679401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e9fdf-600f-447f-b250-5b504f217709 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.691155 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e9fdf-600f-447f-b250-5b504f217709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-167e9fdf-600f-447f-b250-5b504f217709 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:05.696955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:05.697873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:05.698653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 56/114] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:05.699289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e9fdf-600f-447f-b250-5b504f217709 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:05.752773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe16093-aabf-4017-b692-d82217584ece None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:05.800122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec40e18-ce7e-46d5-b7eb-a1f67c8dc047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:05.801100 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec40e18-ce7e-46d5-b7eb-a1f67c8dc047 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.801593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec40e18-ce7e-46d5-b7eb-a1f67c8dc047 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.802124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec40e18-ce7e-46d5-b7eb-a1f67c8dc047 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.815756 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec40e18-ce7e-46d5-b7eb-a1f67c8dc047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:05.818125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e9fdf-600f-447f-b250-5b504f217709 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8ec40e18-ce7e-46d5-b7eb-a1f67c8dc047 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:05.822375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:05.823216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 59/115] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:23:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:05.824495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec40e18-ce7e-46d5-b7eb-a1f67c8dc047 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:05.843922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:05.869289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f4a4a-1515-4cd7-bd33-1df9f690c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:05.870316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f4a4a-1515-4cd7-bd33-1df9f690c550 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.871350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f4a4a-1515-4cd7-bd33-1df9f690c550 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.871813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f4a4a-1515-4cd7-bd33-1df9f690c550 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.887419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f4a4a-1515-4cd7-bd33-1df9f690c550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:05.892734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9e2f4a4a-1515-4cd7-bd33-1df9f690c550 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:05.896639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:05.898098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:05.898886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:05.901154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 57/116] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:05.902342 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f4a4a-1515-4cd7-bd33-1df9f690c550 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:05.949063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec40e18-ce7e-46d5-b7eb-a1f67c8dc047 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:05.989007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4899bc0-f60d-42e8-8ea9-ba9eee45994d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:05.989687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4899bc0-f60d-42e8-8ea9-ba9eee45994d None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:05.989977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4899bc0-f60d-42e8-8ea9-ba9eee45994d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:05.990286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4899bc0-f60d-42e8-8ea9-ba9eee45994d None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe/users/60b0d29f45f6410f8f4e1856e03bc22b/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:05.992874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4899bc0-f60d-42e8-8ea9-ba9eee45994d None admin] RBAC: Authorizing `identity:create_grant(project_id=ce53e46475f341a2af8bf9da4700fffe, user_id=60b0d29f45f6410f8f4e1856e03bc22b, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:05.994339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4899bc0-f60d-42e8-8ea9-ba9eee45994d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:06.036351 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f4a4a-1515-4cd7-bd33-1df9f690c550 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:06.070307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d89da-44ec-437b-b0d7-daac0780fa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:06.070938 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d89da-44ec-437b-b0d7-daac0780fa64 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:06.071224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d89da-44ec-437b-b0d7-daac0780fa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:06.071503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d89da-44ec-437b-b0d7-daac0780fa64 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:06.072249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d89da-44ec-437b-b0d7-daac0780fa64 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:06.072561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d89da-44ec-437b-b0d7-daac0780fa64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:06.073957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d89da-44ec-437b-b0d7-daac0780fa64 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:06.074783 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 60/117] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:23:05 2022] PUT /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe/users/60b0d29f45f6410f8f4e1856e03bc22b/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:06.075660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4899bc0-f60d-42e8-8ea9-ba9eee45994d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:06.083309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 58/118] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:06.092118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d89da-44ec-437b-b0d7-daac0780fa64 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:06.152225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:06.213792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4899bc0-f60d-42e8-8ea9-ba9eee45994d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:06.237468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:06.241928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d89da-44ec-437b-b0d7-daac0780fa64 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:06.251652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c909c1-64c9-471b-a451-a91c38149210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:06.252308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c909c1-64c9-471b-a451-a91c38149210 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:06.252596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c909c1-64c9-471b-a451-a91c38149210 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:06.252872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c909c1-64c9-471b-a451-a91c38149210 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:06.253596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c909c1-64c9-471b-a451-a91c38149210 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:06.253904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c909c1-64c9-471b-a451-a91c38149210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:06.255290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c909c1-64c9-471b-a451-a91c38149210 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:06.267613 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 61/119] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:06.269758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d241a-01a7-4a9c-8ac4-fc5ac5021aaf None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:06.270120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d241a-01a7-4a9c-8ac4-fc5ac5021aaf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:06.270407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d241a-01a7-4a9c-8ac4-fc5ac5021aaf None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:06.271653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 62/120] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:06.273252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d241a-01a7-4a9c-8ac4-fc5ac5021aaf None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:06.288774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731dbf0-bfce-4c9d-abe6-eaafc9c2b12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:06.289617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731dbf0-bfce-4c9d-abe6-eaafc9c2b12b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:06.289903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731dbf0-bfce-4c9d-abe6-eaafc9c2b12b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:06.290179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731dbf0-bfce-4c9d-abe6-eaafc9c2b12b None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:06.290889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731dbf0-bfce-4c9d-abe6-eaafc9c2b12b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:06.291202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731dbf0-bfce-4c9d-abe6-eaafc9c2b12b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:06.292694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731dbf0-bfce-4c9d-abe6-eaafc9c2b12b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:06.301788 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 59/121] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:06.302407 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731dbf0-bfce-4c9d-abe6-eaafc9c2b12b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:06.389799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d241a-01a7-4a9c-8ac4-fc5ac5021aaf None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:06.429016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731dbf0-bfce-4c9d-abe6-eaafc9c2b12b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:06.430615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:06.431205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcefb5-4eeb-4c9f-8109-19380ce8f3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:06.432223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcefb5-4eeb-4c9f-8109-19380ce8f3df None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:06.432712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcefb5-4eeb-4c9f-8109-19380ce8f3df None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:06.433211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcefb5-4eeb-4c9f-8109-19380ce8f3df None admin] PATH_INFO: `/v3/users/ed0dacdea889430f86b9cda4063b3ba9` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:06.450904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcefb5-4eeb-4c9f-8109-19380ce8f3df None admin] RBAC: Authorizing `identity:get_user(user_id=ed0dacdea889430f86b9cda4063b3ba9)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:06.452768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcefb5-4eeb-4c9f-8109-19380ce8f3df None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:06.459475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 63/122] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:06 2022] GET /identity/v3/users/ed0dacdea889430f86b9cda4063b3ba9 => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:06.461743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3046abe-e63b-47d2-a881-7c8d03422ae6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:06.462314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3046abe-e63b-47d2-a881-7c8d03422ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:06.462818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3046abe-e63b-47d2-a881-7c8d03422ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:06.469999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e94164-312c-44a2-93f5-4da8ebebd22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:06.470642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e94164-312c-44a2-93f5-4da8ebebd22f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:06.470923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e94164-312c-44a2-93f5-4da8ebebd22f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:06.471201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e94164-312c-44a2-93f5-4da8ebebd22f None admin] PATH_INFO: `/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:06.472715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e94164-312c-44a2-93f5-4da8ebebd22f None admin] RBAC: Authorizing `identity:get_user(user_id=79d0d7df7a094b5ba4b9a9bb3b449967)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:06.475118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e94164-312c-44a2-93f5-4da8ebebd22f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:06.477115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 60/123] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:06 2022] GET /identity/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:06.477898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e94164-312c-44a2-93f5-4da8ebebd22f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:06.489319 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f3046abe-e63b-47d2-a881-7c8d03422ae6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:06.491727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f3046abe-e63b-47d2-a881-7c8d03422ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:06.529437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:06.567394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 64/124] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:06.570287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3046abe-e63b-47d2-a881-7c8d03422ae6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:06.609582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e94164-312c-44a2-93f5-4da8ebebd22f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:06.645613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c7b54-83ed-49be-88a3-eda56355d6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:06.646652 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c7b54-83ed-49be-88a3-eda56355d6dd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:06.647702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c7b54-83ed-49be-88a3-eda56355d6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:06.648229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c7b54-83ed-49be-88a3-eda56355d6dd None admin] PATH_INFO: `/v3/users/60b0d29f45f6410f8f4e1856e03bc22b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:06.649637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c7b54-83ed-49be-88a3-eda56355d6dd None admin] RBAC: Authorizing `identity:get_user(user_id=60b0d29f45f6410f8f4e1856e03bc22b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:06.651398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c7b54-83ed-49be-88a3-eda56355d6dd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:06.653223 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 61/125] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:06 2022] GET /identity/v3/users/60b0d29f45f6410f8f4e1856e03bc22b => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:06.654172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c7b54-83ed-49be-88a3-eda56355d6dd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:06.698766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:06.701820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3046abe-e63b-47d2-a881-7c8d03422ae6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:06.748838 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fe191-f5a1-4bbb-a8fd-63a31cfa4380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:06.749511 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe191-f5a1-4bbb-a8fd-63a31cfa4380 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:06.750204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe191-f5a1-4bbb-a8fd-63a31cfa4380 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:06.750496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe191-f5a1-4bbb-a8fd-63a31cfa4380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:06.768922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c7b54-83ed-49be-88a3-eda56355d6dd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:06.808269 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:06.809900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbaa2e0-7db4-422f-9634-d8f79fc432a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:06.810548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbaa2e0-7db4-422f-9634-d8f79fc432a5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:06.810831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbaa2e0-7db4-422f-9634-d8f79fc432a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:06.811112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbaa2e0-7db4-422f-9634-d8f79fc432a5 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:06.813780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbaa2e0-7db4-422f-9634-d8f79fc432a5 None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:06.815226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbaa2e0-7db4-422f-9634-d8f79fc432a5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:06.822980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 62/126] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:06 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:06.823451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbaa2e0-7db4-422f-9634-d8f79fc432a5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:06.846718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fe191-f5a1-4bbb-a8fd-63a31cfa4380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:06.849731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fe191-f5a1-4bbb-a8fd-63a31cfa4380 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:06.957177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbaa2e0-7db4-422f-9634-d8f79fc432a5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:06.961851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 65/127] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:06.962767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fe191-f5a1-4bbb-a8fd-63a31cfa4380 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:06.968112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:07.005666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc25794-3155-406a-8998-3df9284b7297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:07.006305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc25794-3155-406a-8998-3df9284b7297 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:07.006977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc25794-3155-406a-8998-3df9284b7297 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:07.006977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc25794-3155-406a-8998-3df9284b7297 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:07.017508 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc25794-3155-406a-8998-3df9284b7297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bcc25794-3155-406a-8998-3df9284b7297 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:07.022741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:07.023581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 63/128] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:07.029379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc25794-3155-406a-8998-3df9284b7297 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:07.052490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:07.093963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fe191-f5a1-4bbb-a8fd-63a31cfa4380 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:07.135514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc25794-3155-406a-8998-3df9284b7297 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:07.140573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a271150-b4b3-4ac4-b6a8-c6544b4b6b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:07.141516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a271150-b4b3-4ac4-b6a8-c6544b4b6b5e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:07.142002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a271150-b4b3-4ac4-b6a8-c6544b4b6b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:07.142475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a271150-b4b3-4ac4-b6a8-c6544b4b6b5e None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:07.144400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a271150-b4b3-4ac4-b6a8-c6544b4b6b5e None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:07.146970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a271150-b4b3-4ac4-b6a8-c6544b4b6b5e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:07.149776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 66/129] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:06 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:07.151356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a271150-b4b3-4ac4-b6a8-c6544b4b6b5e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:07.187147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713e51-c709-4e1d-91ea-3a85e9d1b193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:07.188368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713e51-c709-4e1d-91ea-3a85e9d1b193 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:07.194898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713e51-c709-4e1d-91ea-3a85e9d1b193 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:07.194898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713e51-c709-4e1d-91ea-3a85e9d1b193 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe/users/79d0d7df7a094b5ba4b9a9bb3b449967/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:07.194898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c713e51-c709-4e1d-91ea-3a85e9d1b193 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce53e46475f341a2af8bf9da4700fffe, user_id=79d0d7df7a094b5ba4b9a9bb3b449967, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:07.194898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c713e51-c709-4e1d-91ea-3a85e9d1b193 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:07.268697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a271150-b4b3-4ac4-b6a8-c6544b4b6b5e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:07.276662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 64/130] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:23:07 2022] PUT /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe/users/79d0d7df7a094b5ba4b9a9bb3b449967/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:07.277658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713e51-c709-4e1d-91ea-3a85e9d1b193 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:07.322167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:07.352813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ddcc-bfdb-41a5-b134-6fc45c507bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:07.353391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ddcc-bfdb-41a5-b134-6fc45c507bfd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:07.353676 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ddcc-bfdb-41a5-b134-6fc45c507bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:07.353951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ddcc-bfdb-41a5-b134-6fc45c507bfd None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:07.355166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926ddcc-bfdb-41a5-b134-6fc45c507bfd None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:07.356635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926ddcc-bfdb-41a5-b134-6fc45c507bfd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:07.358638 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 67/131] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:07 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:07.359487 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ddcc-bfdb-41a5-b134-6fc45c507bfd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:07.379600 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:07.419934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713e51-c709-4e1d-91ea-3a85e9d1b193 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:07.464862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429ffd1-ecd8-4310-8f26-624a94324390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:07.465559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ffd1-ecd8-4310-8f26-624a94324390 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:07.465939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ffd1-ecd8-4310-8f26-624a94324390 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:07.466234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ffd1-ecd8-4310-8f26-624a94324390 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:07.467071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429ffd1-ecd8-4310-8f26-624a94324390 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:07.467383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429ffd1-ecd8-4310-8f26-624a94324390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:07.469162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429ffd1-ecd8-4310-8f26-624a94324390 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:07.480118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 65/132] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:07.481110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429ffd1-ecd8-4310-8f26-624a94324390 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:07.591033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ddcc-bfdb-41a5-b134-6fc45c507bfd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:07.601467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:07.619741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429ffd1-ecd8-4310-8f26-624a94324390 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:07.639070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2416c52-702e-48a8-955c-627c430360a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:07.639897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2416c52-702e-48a8-955c-627c430360a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:07.640623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2416c52-702e-48a8-955c-627c430360a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:07.640906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2416c52-702e-48a8-955c-627c430360a8 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe/users/60b0d29f45f6410f8f4e1856e03bc22b/roles/e3c5a858a45c41db8ff3e883788d75bc` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:07.643551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2416c52-702e-48a8-955c-627c430360a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce53e46475f341a2af8bf9da4700fffe, user_id=60b0d29f45f6410f8f4e1856e03bc22b, role_id=e3c5a858a45c41db8ff3e883788d75bc)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:07.645412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2416c52-702e-48a8-955c-627c430360a8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:07.668087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41361bfd-6d09-4a3d-a528-cac6054b2821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:07.668718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361bfd-6d09-4a3d-a528-cac6054b2821 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:07.669054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361bfd-6d09-4a3d-a528-cac6054b2821 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:07.669341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361bfd-6d09-4a3d-a528-cac6054b2821 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:07.670144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41361bfd-6d09-4a3d-a528-cac6054b2821 None admin] RBAC: Adding query filter params (role.id=e2331f0635214139bdac452081a3f310, scope.project.id=ce53e46475f341a2af8bf9da4700fffe, user.id=ed0dacdea889430f86b9cda4063b3ba9) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:07.670462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41361bfd-6d09-4a3d-a528-cac6054b2821 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:07.672165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41361bfd-6d09-4a3d-a528-cac6054b2821 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:07.679305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 66/133] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:07 2022] GET /identity/v3/role_assignments?user.id=ed0dacdea889430f86b9cda4063b3ba9&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=e2331f0635214139bdac452081a3f310 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:07.680404 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41361bfd-6d09-4a3d-a528-cac6054b2821 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:07.705370 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 68/134] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:23:07 2022] PUT /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe/users/60b0d29f45f6410f8f4e1856e03bc22b/roles/e3c5a858a45c41db8ff3e883788d75bc => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:07.810946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2a6c-47de-46e9-b80d-b0400b003521 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:07.811716 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2a6c-47de-46e9-b80d-b0400b003521 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:07.811716 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2a6c-47de-46e9-b80d-b0400b003521 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:07.813207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 69/135] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:23:07.819774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7bea4-3453-4754-9c9b-782d637ce935 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:07.820771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7bea4-3453-4754-9c9b-782d637ce935 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:07.820938 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7bea4-3453-4754-9c9b-782d637ce935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:07.827482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41361bfd-6d09-4a3d-a528-cac6054b2821 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:07.842209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dfe7bea4-3453-4754-9c9b-782d637ce935 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:07.843525 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dfe7bea4-3453-4754-9c9b-782d637ce935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:07.906465 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce367c-d19d-4c4c-a68b-2a7b1ea65b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:07.907513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce367c-d19d-4c4c-a68b-2a7b1ea65b85 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:07.908066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce367c-d19d-4c4c-a68b-2a7b1ea65b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:07.908682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce367c-d19d-4c4c-a68b-2a7b1ea65b85 None admin] PATH_INFO: `/v3/users/60b0d29f45f6410f8f4e1856e03bc22b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:07.912858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce367c-d19d-4c4c-a68b-2a7b1ea65b85 None admin] RBAC: Authorizing `identity:get_user(user_id=60b0d29f45f6410f8f4e1856e03bc22b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:07.914755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce367c-d19d-4c4c-a68b-2a7b1ea65b85 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:07.916746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 67/136] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:07 2022] GET /identity/v3/users/60b0d29f45f6410f8f4e1856e03bc22b => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:07.922691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce367c-d19d-4c4c-a68b-2a7b1ea65b85 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:07.927785 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:07.937531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 70/137] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:07.966762 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:07.969939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7bea4-3453-4754-9c9b-782d637ce935 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:08.083654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7bea4-3453-4754-9c9b-782d637ce935 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:08.122686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245e189-40b3-44c5-97b8-31418ef1d707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:08.123349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245e189-40b3-44c5-97b8-31418ef1d707 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:08.123637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245e189-40b3-44c5-97b8-31418ef1d707 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:08.123917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245e189-40b3-44c5-97b8-31418ef1d707 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:08.125178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245e189-40b3-44c5-97b8-31418ef1d707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:08.126636 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245e189-40b3-44c5-97b8-31418ef1d707 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:08.128572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 71/138] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:08.134166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245e189-40b3-44c5-97b8-31418ef1d707 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:08.159747 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce367c-d19d-4c4c-a68b-2a7b1ea65b85 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:08.202809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:08.245215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245e189-40b3-44c5-97b8-31418ef1d707 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:08.264698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70f06a-e5d5-46a8-82da-b7e02d3dc492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:08.265366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70f06a-e5d5-46a8-82da-b7e02d3dc492 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:08.265658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70f06a-e5d5-46a8-82da-b7e02d3dc492 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:08.265943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70f06a-e5d5-46a8-82da-b7e02d3dc492 None admin] PATH_INFO: `/v3/projects/6c34cca62b0d4a439cd929a3b9f9de41` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:08.267156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70f06a-e5d5-46a8-82da-b7e02d3dc492 None admin] RBAC: Authorizing `identity:get_project(project_id=6c34cca62b0d4a439cd929a3b9f9de41)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:08.268659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70f06a-e5d5-46a8-82da-b7e02d3dc492 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:08.270539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 68/139] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:07 2022] GET /identity/v3/projects/6c34cca62b0d4a439cd929a3b9f9de41 => generated 312 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:08.276451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70f06a-e5d5-46a8-82da-b7e02d3dc492 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:08.300608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9d362-0be7-4570-805e-91e7f5e313b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:08.301240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9d362-0be7-4570-805e-91e7f5e313b6 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:08.301528 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9d362-0be7-4570-805e-91e7f5e313b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:08.301810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9d362-0be7-4570-805e-91e7f5e313b6 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:08.302512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9d362-0be7-4570-805e-91e7f5e313b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:08.308701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9d362-0be7-4570-805e-91e7f5e313b6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:08.417175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:08.429304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70f06a-e5d5-46a8-82da-b7e02d3dc492 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:08.439773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 72/140] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:23:08 2022] POST /identity/v3/users => generated 283 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:23:08.485873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18e602-4a60-4b8e-ad7f-6e0f437c0aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:08.486709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18e602-4a60-4b8e-ad7f-6e0f437c0aeb None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:08.486995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18e602-4a60-4b8e-ad7f-6e0f437c0aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:08.487279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18e602-4a60-4b8e-ad7f-6e0f437c0aeb None admin] PATH_INFO: `/v3/projects/6c34cca62b0d4a439cd929a3b9f9de41/users/60b0d29f45f6410f8f4e1856e03bc22b/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:08.491063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18e602-4a60-4b8e-ad7f-6e0f437c0aeb None admin] RBAC: Authorizing `identity:create_grant(project_id=6c34cca62b0d4a439cd929a3b9f9de41, user_id=60b0d29f45f6410f8f4e1856e03bc22b, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:08.492663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18e602-4a60-4b8e-ad7f-6e0f437c0aeb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:08.577208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 69/141] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:23:08 2022] PUT /identity/v3/projects/6c34cca62b0d4a439cd929a3b9f9de41/users/60b0d29f45f6410f8f4e1856e03bc22b/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:23:08.754533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d120-f75a-4359-aaa6-6498c0762255 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:08.755052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d120-f75a-4359-aaa6-6498c0762255 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:08.755158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d120-f75a-4359-aaa6-6498c0762255 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:08.757113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 73/142] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:08.762874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1375-f221-42f8-81c4-7cc154c30573 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:08.763349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1375-f221-42f8-81c4-7cc154c30573 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:08.763429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1375-f221-42f8-81c4-7cc154c30573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:08.787357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7f4e1375-f221-42f8-81c4-7cc154c30573 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:08.789101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7f4e1375-f221-42f8-81c4-7cc154c30573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:08.878534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 70/143] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:08.931889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484d120-f75a-4359-aaa6-6498c0762255 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:09.059555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484d120-f75a-4359-aaa6-6498c0762255 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:09.083046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:09.103932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dca3fa-9c7f-418b-ac16-c600722f9951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:09.104629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dca3fa-9c7f-418b-ac16-c600722f9951 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:09.104920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dca3fa-9c7f-418b-ac16-c600722f9951 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:09.120322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dca3fa-9c7f-418b-ac16-c600722f9951 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:09.123444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dca3fa-9c7f-418b-ac16-c600722f9951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f8dca3fa-9c7f-418b-ac16-c600722f9951 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:09.128729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:09.129597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:09.130350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:09.132723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 74/144] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:09.136057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e1375-f221-42f8-81c4-7cc154c30573 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:09.227490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:09.251606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e1375-f221-42f8-81c4-7cc154c30573 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:09.288665 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d456e3b-853c-4337-a66d-a82357e267a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:09.289288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d456e3b-853c-4337-a66d-a82357e267a3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:09.289574 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d456e3b-853c-4337-a66d-a82357e267a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:09.289851 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d456e3b-853c-4337-a66d-a82357e267a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:09.290570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d456e3b-853c-4337-a66d-a82357e267a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:09.290880 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d456e3b-853c-4337-a66d-a82357e267a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:09.292288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d456e3b-853c-4337-a66d-a82357e267a3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:09.301247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 71/145] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:09.307591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dca3fa-9c7f-418b-ac16-c600722f9951 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:09.439748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dca3fa-9c7f-418b-ac16-c600722f9951 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:09.480668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477fb42-32e8-474d-a77e-23540ae677d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:09.481787 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477fb42-32e8-474d-a77e-23540ae677d6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:09.486610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477fb42-32e8-474d-a77e-23540ae677d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:09.487124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477fb42-32e8-474d-a77e-23540ae677d6 None admin] PATH_INFO: `/v3/users/60b0d29f45f6410f8f4e1856e03bc22b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:09.495628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477fb42-32e8-474d-a77e-23540ae677d6 None admin] RBAC: Authorizing `identity:get_user(user_id=60b0d29f45f6410f8f4e1856e03bc22b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:09.497453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477fb42-32e8-474d-a77e-23540ae677d6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:09.503275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 75/146] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:09 2022] GET /identity/v3/users/60b0d29f45f6410f8f4e1856e03bc22b => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:09.509891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d456e3b-853c-4337-a66d-a82357e267a3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:09.662610 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d456e3b-853c-4337-a66d-a82357e267a3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:09.705429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f75c6-000f-48c8-8061-15d38dc2bc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:09.705984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f75c6-000f-48c8-8061-15d38dc2bc18 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:09.706266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f75c6-000f-48c8-8061-15d38dc2bc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:09.706539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f75c6-000f-48c8-8061-15d38dc2bc18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:09.717826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:09.790526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f75c6-000f-48c8-8061-15d38dc2bc18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:09.792032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f75c6-000f-48c8-8061-15d38dc2bc18 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:09.920175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 72/147] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:09.921521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477fb42-32e8-474d-a77e-23540ae677d6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:10.003278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:10.058026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477fb42-32e8-474d-a77e-23540ae677d6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:10.096431 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc358f60-5a99-4530-963a-1ab2d14a681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:10.097095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc358f60-5a99-4530-963a-1ab2d14a681c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:10.097381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc358f60-5a99-4530-963a-1ab2d14a681c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:10.097659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc358f60-5a99-4530-963a-1ab2d14a681c None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:10.101578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc358f60-5a99-4530-963a-1ab2d14a681c None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:10.103057 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc358f60-5a99-4530-963a-1ab2d14a681c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:10.105193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 76/148] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:09 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:10.111062 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f75c6-000f-48c8-8061-15d38dc2bc18 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:10.167494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:10.248220 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f75c6-000f-48c8-8061-15d38dc2bc18 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:10.290359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccafee-f47d-4293-9afb-054c73deadf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:10.290977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccafee-f47d-4293-9afb-054c73deadf7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:10.291256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccafee-f47d-4293-9afb-054c73deadf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:10.291528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccafee-f47d-4293-9afb-054c73deadf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:10.292346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccafee-f47d-4293-9afb-054c73deadf7 None admin] RBAC: Adding query filter params (role.id=525e499f44594268b4f362c62bd35e00, scope.project.id=ce53e46475f341a2af8bf9da4700fffe, user.id=60b0d29f45f6410f8f4e1856e03bc22b) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:10.292657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccafee-f47d-4293-9afb-054c73deadf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:10.294111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccafee-f47d-4293-9afb-054c73deadf7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:10.305385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 73/149] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:10 2022] GET /identity/v3/role_assignments?user.id=60b0d29f45f6410f8f4e1856e03bc22b&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=525e499f44594268b4f362c62bd35e00 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:10.320000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e85a48-ac40-466d-86b4-54f522a345fb None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:10.320617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e85a48-ac40-466d-86b4-54f522a345fb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:10.321100 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e85a48-ac40-466d-86b4-54f522a345fb None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:10.322805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 77/150] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:10.328541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb969b3-efa3-490a-b091-44e5db40805f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:10.328827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb969b3-efa3-490a-b091-44e5db40805f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:10.329105 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb969b3-efa3-490a-b091-44e5db40805f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:10.353293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3cb969b3-efa3-490a-b091-44e5db40805f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:10.354481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3cb969b3-efa3-490a-b091-44e5db40805f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:10.435729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 74/151] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:10.466722 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e85a48-ac40-466d-86b4-54f522a345fb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:10.615020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e85a48-ac40-466d-86b4-54f522a345fb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:10.637999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:10.659697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eae9af-e4ef-44d1-8bf4-c4d6a59d4684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:10.660380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eae9af-e4ef-44d1-8bf4-c4d6a59d4684 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:10.660669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eae9af-e4ef-44d1-8bf4-c4d6a59d4684 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:10.660987 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eae9af-e4ef-44d1-8bf4-c4d6a59d4684 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:10.674514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eae9af-e4ef-44d1-8bf4-c4d6a59d4684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f3eae9af-e4ef-44d1-8bf4-c4d6a59d4684 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:10.680287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:10.681166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:10.681929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 78/152] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:10.687562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb969b3-efa3-490a-b091-44e5db40805f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:10.748775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:10.827034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb969b3-efa3-490a-b091-44e5db40805f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:10.872657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd6c51-5a20-40eb-9510-94561a5cdf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:10.873204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd6c51-5a20-40eb-9510-94561a5cdf76 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:10.873483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd6c51-5a20-40eb-9510-94561a5cdf76 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:10.873753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd6c51-5a20-40eb-9510-94561a5cdf76 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:10.874486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd6c51-5a20-40eb-9510-94561a5cdf76 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:10.874792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd6c51-5a20-40eb-9510-94561a5cdf76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:10.876243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd6c51-5a20-40eb-9510-94561a5cdf76 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:10.889166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 75/153] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:10.894784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eae9af-e4ef-44d1-8bf4-c4d6a59d4684 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:10.983426 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2cae9-cbf5-4b77-b5f6-3b2686029744 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:10.986190 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2cae9-cbf5-4b77-b5f6-3b2686029744 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:10.986669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2cae9-cbf5-4b77-b5f6-3b2686029744 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:10.988559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 76/154] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:11.002668 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b81b0-8b2f-47b8-99fd-ceaccc069937 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:11.003272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b81b0-8b2f-47b8-99fd-ceaccc069937 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:11.003748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b81b0-8b2f-47b8-99fd-ceaccc069937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:11.043896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9b4b81b0-8b2f-47b8-99fd-ceaccc069937 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:11.046177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9b4b81b0-8b2f-47b8-99fd-ceaccc069937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:11.054786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eae9af-e4ef-44d1-8bf4-c4d6a59d4684 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:11.137473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012aa18-e633-4375-aa37-7941a62520eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:11.138058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012aa18-e633-4375-aa37-7941a62520eb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:11.138505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012aa18-e633-4375-aa37-7941a62520eb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:11.138926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012aa18-e633-4375-aa37-7941a62520eb None admin] PATH_INFO: `/v3/users/ed0dacdea889430f86b9cda4063b3ba9` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:11.140359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012aa18-e633-4375-aa37-7941a62520eb None admin] RBAC: Authorizing `identity:get_user(user_id=ed0dacdea889430f86b9cda4063b3ba9)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:11.141972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012aa18-e633-4375-aa37-7941a62520eb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:11.143577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 79/155] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:10 2022] GET /identity/v3/users/ed0dacdea889430f86b9cda4063b3ba9 => generated 289 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:11.149752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012aa18-e633-4375-aa37-7941a62520eb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:11.153699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 77/156] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:11.182753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b81b0-8b2f-47b8-99fd-ceaccc069937 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:11.220400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:11.227241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:11.296905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012aa18-e633-4375-aa37-7941a62520eb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:11.326007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b81b0-8b2f-47b8-99fd-ceaccc069937 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:11.362223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47766b-e5ab-4444-ae20-640c053f9ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:11.363910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47766b-e5ab-4444-ae20-640c053f9ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:11.364356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47766b-e5ab-4444-ae20-640c053f9ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:11.364807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47766b-e5ab-4444-ae20-640c053f9ba7 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:11.366299 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47766b-e5ab-4444-ae20-640c053f9ba7 None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:11.368102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47766b-e5ab-4444-ae20-640c053f9ba7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:11.370393 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 80/157] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:11 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:11.375708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47766b-e5ab-4444-ae20-640c053f9ba7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:11.398592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130bb1a-7ee9-4544-af83-2a2b5ccb12cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:11.400019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130bb1a-7ee9-4544-af83-2a2b5ccb12cb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:11.400651 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130bb1a-7ee9-4544-af83-2a2b5ccb12cb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:11.401806 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130bb1a-7ee9-4544-af83-2a2b5ccb12cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:11.422730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130bb1a-7ee9-4544-af83-2a2b5ccb12cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d130bb1a-7ee9-4544-af83-2a2b5ccb12cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:11.429134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:11.430026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:11.430809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:11.437359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 78/158] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:23:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:11.439923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130bb1a-7ee9-4544-af83-2a2b5ccb12cb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:11.553963 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:11.570496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130bb1a-7ee9-4544-af83-2a2b5ccb12cb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:11.608367 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e2851-d554-43cb-b62c-b9b931e0a246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:11.609091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e2851-d554-43cb-b62c-b9b931e0a246 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:11.609452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e2851-d554-43cb-b62c-b9b931e0a246 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:11.609796 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e2851-d554-43cb-b62c-b9b931e0a246 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:11.616012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e2851-d554-43cb-b62c-b9b931e0a246 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:11.616012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e2851-d554-43cb-b62c-b9b931e0a246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:11.616012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e2851-d554-43cb-b62c-b9b931e0a246 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:11.619562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47766b-e5ab-4444-ae20-640c053f9ba7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:11.624156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 79/159] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:11.624291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8a21-73b0-4f52-9f0f-7ec5359d4460 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:11.624633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8a21-73b0-4f52-9f0f-7ec5359d4460 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:11.624986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8a21-73b0-4f52-9f0f-7ec5359d4460 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:11.626269 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 80/160] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:11.632078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe65ff-009d-42f4-af99-b71ad5ce80ba None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:11.632436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe65ff-009d-42f4-af99-b71ad5ce80ba None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:11.632821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe65ff-009d-42f4-af99-b71ad5ce80ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:11.640411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:11.652749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-27fe65ff-009d-42f4-af99-b71ad5ce80ba None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:11.653943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-27fe65ff-009d-42f4-af99-b71ad5ce80ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:11.664486 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ab49b-5413-4204-9963-046e1d1b6601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:11.665412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ab49b-5413-4204-9963-046e1d1b6601 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:11.665922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ab49b-5413-4204-9963-046e1d1b6601 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:11.667136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ab49b-5413-4204-9963-046e1d1b6601 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe/users/ed0dacdea889430f86b9cda4063b3ba9/roles/e2331f0635214139bdac452081a3f310` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:11.669674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ab49b-5413-4204-9963-046e1d1b6601 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce53e46475f341a2af8bf9da4700fffe, user_id=ed0dacdea889430f86b9cda4063b3ba9, role_id=e2331f0635214139bdac452081a3f310)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:11.671426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ab49b-5413-4204-9963-046e1d1b6601 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:11.733933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 81/161] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:11.735249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe65ff-009d-42f4-af99-b71ad5ce80ba None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:11.758930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 81/162] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:23:11 2022] PUT /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe/users/ed0dacdea889430f86b9cda4063b3ba9/roles/e2331f0635214139bdac452081a3f310 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:23:11.785830 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ab49b-5413-4204-9963-046e1d1b6601 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:11.832189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:11.858486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe65ff-009d-42f4-af99-b71ad5ce80ba None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:11.901046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbb665-0525-487b-b368-1387acc216b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:11.901738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbb665-0525-487b-b368-1387acc216b8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:11.902567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbb665-0525-487b-b368-1387acc216b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:11.902906 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbb665-0525-487b-b368-1387acc216b8 None admin] PATH_INFO: `/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:11.904378 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbb665-0525-487b-b368-1387acc216b8 None admin] RBAC: Authorizing `identity:get_user(user_id=79d0d7df7a094b5ba4b9a9bb3b449967)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:11.906003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbb665-0525-487b-b368-1387acc216b8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:11.907836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 82/163] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:11 2022] GET /identity/v3/users/79d0d7df7a094b5ba4b9a9bb3b449967 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:11.917694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbb665-0525-487b-b368-1387acc216b8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:12.015295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ab49b-5413-4204-9963-046e1d1b6601 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:12.045650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbb665-0525-487b-b368-1387acc216b8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:12.089440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a814ca7-e1e1-4e72-919c-709f0758e17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:12.090077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a814ca7-e1e1-4e72-919c-709f0758e17b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:12.090361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a814ca7-e1e1-4e72-919c-709f0758e17b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:12.090639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a814ca7-e1e1-4e72-919c-709f0758e17b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:12.095874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3b95c-d180-45a8-bb94-e0228e043339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:12.096554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3b95c-d180-45a8-bb94-e0228e043339 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:12.096836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3b95c-d180-45a8-bb94-e0228e043339 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:12.097114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3b95c-d180-45a8-bb94-e0228e043339 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:12.123967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3b95c-d180-45a8-bb94-e0228e043339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-95d3b95c-d180-45a8-bb94-e0228e043339 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:12.129428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:12.130354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:12.131606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 82/164] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:12.132808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:12.140952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3b95c-d180-45a8-bb94-e0228e043339 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:12.210925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a814ca7-e1e1-4e72-919c-709f0758e17b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:12.212484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a814ca7-e1e1-4e72-919c-709f0758e17b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:12.236392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:12.324770 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 83/165] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:12.333567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a814ca7-e1e1-4e72-919c-709f0758e17b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:12.342846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3b95c-d180-45a8-bb94-e0228e043339 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:12.383225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:12.401366 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cce87d-32d1-4462-b702-c88a52b9f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:12.402174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cce87d-32d1-4462-b702-c88a52b9f194 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:12.402947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cce87d-32d1-4462-b702-c88a52b9f194 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:12.403310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cce87d-32d1-4462-b702-c88a52b9f194 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:12.404291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cce87d-32d1-4462-b702-c88a52b9f194 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:12.404803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cce87d-32d1-4462-b702-c88a52b9f194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:12.406433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cce87d-32d1-4462-b702-c88a52b9f194 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:12.416447 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 83/166] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:12.425102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cce87d-32d1-4462-b702-c88a52b9f194 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:12.453725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a814ca7-e1e1-4e72-919c-709f0758e17b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:12.520432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d7aa-d5c4-44fa-9c9e-57c6ed7d3e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:12.521302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d7aa-d5c4-44fa-9c9e-57c6ed7d3e16 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:12.521769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d7aa-d5c4-44fa-9c9e-57c6ed7d3e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:12.522925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d7aa-d5c4-44fa-9c9e-57c6ed7d3e16 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:12.525053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d7aa-d5c4-44fa-9c9e-57c6ed7d3e16 None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:12.529372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d7aa-d5c4-44fa-9c9e-57c6ed7d3e16 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:12.531257 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 84/167] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:12 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:12.539073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d7aa-d5c4-44fa-9c9e-57c6ed7d3e16 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:12.590147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:12.605019 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cce87d-32d1-4462-b702-c88a52b9f194 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:12.665829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2615f-3aa7-4ed4-93cb-a9750b0ac1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:12.667837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2615f-3aa7-4ed4-93cb-a9750b0ac1cb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:12.668515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2615f-3aa7-4ed4-93cb-a9750b0ac1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:12.669022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2615f-3aa7-4ed4-93cb-a9750b0ac1cb None admin] PATH_INFO: `/v3/users/60b0d29f45f6410f8f4e1856e03bc22b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:12.672707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2615f-3aa7-4ed4-93cb-a9750b0ac1cb None admin] RBAC: Authorizing `identity:get_user(user_id=60b0d29f45f6410f8f4e1856e03bc22b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:12.674375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2615f-3aa7-4ed4-93cb-a9750b0ac1cb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:12.677695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 84/168] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:12 2022] GET /identity/v3/users/60b0d29f45f6410f8f4e1856e03bc22b => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:12.684171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2615f-3aa7-4ed4-93cb-a9750b0ac1cb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:12.733677 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d7aa-d5c4-44fa-9c9e-57c6ed7d3e16 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:12.761649 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:12.797964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21009b3d-5d60-4b32-9d6a-a3e9f8b6e2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:12.798959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009b3d-5d60-4b32-9d6a-a3e9f8b6e2ec None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:12.799477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009b3d-5d60-4b32-9d6a-a3e9f8b6e2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:12.800559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21009b3d-5d60-4b32-9d6a-a3e9f8b6e2ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:12.802121 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21009b3d-5d60-4b32-9d6a-a3e9f8b6e2ec None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=ce53e46475f341a2af8bf9da4700fffe, user.id=79d0d7df7a094b5ba4b9a9bb3b449967) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:12.808678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21009b3d-5d60-4b32-9d6a-a3e9f8b6e2ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:12.810461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21009b3d-5d60-4b32-9d6a-a3e9f8b6e2ec None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:12.828016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 85/169] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:12 2022] GET /identity/v3/role_assignments?user.id=79d0d7df7a094b5ba4b9a9bb3b449967&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 608 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:12.828016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8b8bb-e44e-4ca9-b0f1-2b741b1c0c89 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:12.828016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8b8bb-e44e-4ca9-b0f1-2b741b1c0c89 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:12.828016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8b8bb-e44e-4ca9-b0f1-2b741b1c0c89 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:12.832788 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 86/170] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:12.839872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca6348-ef11-4b54-8c2e-c5ea3dc06fcd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:12.840232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca6348-ef11-4b54-8c2e-c5ea3dc06fcd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:12.840511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca6348-ef11-4b54-8c2e-c5ea3dc06fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:12.863789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2615f-3aa7-4ed4-93cb-a9750b0ac1cb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:12.888049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4cca6348-ef11-4b54-8c2e-c5ea3dc06fcd None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:12.888049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4cca6348-ef11-4b54-8c2e-c5ea3dc06fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:12.925889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea1c35-e1b4-4c17-8db0-b4dc17a67388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:12.926885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea1c35-e1b4-4c17-8db0-b4dc17a67388 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:12.927374 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea1c35-e1b4-4c17-8db0-b4dc17a67388 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:12.927851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea1c35-e1b4-4c17-8db0-b4dc17a67388 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.023669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 87/171] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:13.035762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:13.100784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca6348-ef11-4b54-8c2e-c5ea3dc06fcd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:13.111655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea1c35-e1b4-4c17-8db0-b4dc17a67388 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:13.113166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea1c35-e1b4-4c17-8db0-b4dc17a67388 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:13.264794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca6348-ef11-4b54-8c2e-c5ea3dc06fcd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:13.284709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:13.332639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce64385-e77d-4884-909c-1eda46e8e74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:13.333291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce64385-e77d-4884-909c-1eda46e8e74f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:13.333570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce64385-e77d-4884-909c-1eda46e8e74f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:13.333843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce64385-e77d-4884-909c-1eda46e8e74f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.348684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 85/172] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:13.353981 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea1c35-e1b4-4c17-8db0-b4dc17a67388 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:13.356893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce64385-e77d-4884-909c-1eda46e8e74f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9ce64385-e77d-4884-909c-1eda46e8e74f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:13.362283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:13.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:13.364236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:13.368448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 88/173] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:13.374344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce64385-e77d-4884-909c-1eda46e8e74f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:13.452432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:13.502141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea1c35-e1b4-4c17-8db0-b4dc17a67388 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:13.546973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a05d5e-e322-4f27-8fc1-d6c2c368719f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:13.547615 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a05d5e-e322-4f27-8fc1-d6c2c368719f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:13.547896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a05d5e-e322-4f27-8fc1-d6c2c368719f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:13.548347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a05d5e-e322-4f27-8fc1-d6c2c368719f None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.549889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a05d5e-e322-4f27-8fc1-d6c2c368719f None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:13.551762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a05d5e-e322-4f27-8fc1-d6c2c368719f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:13.554863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 86/174] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:13 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:13.556119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059730c-06a7-416e-af23-6625a5f6c4a3 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:13.556404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059730c-06a7-416e-af23-6625a5f6c4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:13.556677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059730c-06a7-416e-af23-6625a5f6c4a3 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.558055 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 87/175] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:13.562842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059730c-06a7-416e-af23-6625a5f6c4a3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:13.580377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce64385-e77d-4884-909c-1eda46e8e74f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:13.641053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34858ba0-91c3-4c1c-9e1f-3f5cf21206d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:13.641651 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34858ba0-91c3-4c1c-9e1f-3f5cf21206d4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:13.641933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34858ba0-91c3-4c1c-9e1f-3f5cf21206d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:13.642204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34858ba0-91c3-4c1c-9e1f-3f5cf21206d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.643172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34858ba0-91c3-4c1c-9e1f-3f5cf21206d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:13.643480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34858ba0-91c3-4c1c-9e1f-3f5cf21206d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:13.645755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34858ba0-91c3-4c1c-9e1f-3f5cf21206d4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:13.657613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 89/176] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:13.659122 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe338-9b90-4c8d-841a-5dac482662d6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:13.659407 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe338-9b90-4c8d-841a-5dac482662d6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:13.659940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe338-9b90-4c8d-841a-5dac482662d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.682687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2e6fe338-9b90-4c8d-841a-5dac482662d6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:13.683997 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2e6fe338-9b90-4c8d-841a-5dac482662d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:13.689426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059730c-06a7-416e-af23-6625a5f6c4a3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:13.708730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:13.721580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:13.727823 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c58d53-0051-4b88-aa1d-df4da94174a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:13.728482 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c58d53-0051-4b88-aa1d-df4da94174a6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:13.728772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c58d53-0051-4b88-aa1d-df4da94174a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:13.729056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c58d53-0051-4b88-aa1d-df4da94174a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.729870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c58d53-0051-4b88-aa1d-df4da94174a6 None admin] RBAC: Adding query filter params (role.id=e3c5a858a45c41db8ff3e883788d75bc, scope.project.id=ce53e46475f341a2af8bf9da4700fffe, user.id=60b0d29f45f6410f8f4e1856e03bc22b) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:13.730242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c58d53-0051-4b88-aa1d-df4da94174a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:13.731697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c58d53-0051-4b88-aa1d-df4da94174a6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:13.737842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 88/177] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:13 2022] GET /identity/v3/role_assignments?user.id=60b0d29f45f6410f8f4e1856e03bc22b&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=e3c5a858a45c41db8ff3e883788d75bc => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:13.739091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba0d2f-d5df-4246-b517-dd2e6dbf9072 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:13.739383 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba0d2f-d5df-4246-b517-dd2e6dbf9072 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:13.739663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba0d2f-d5df-4246-b517-dd2e6dbf9072 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.741714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 89/178] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:13.744826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba0d2f-d5df-4246-b517-dd2e6dbf9072 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:13.773515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 90/179] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:13.773515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459890b-df89-463b-a407-5cee0ac67b48 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:13.773515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459890b-df89-463b-a407-5cee0ac67b48 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:13.773515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459890b-df89-463b-a407-5cee0ac67b48 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.773515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 91/180] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:13.773515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7345c-84e5-435f-a271-33b25993a459 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:13.774033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7345c-84e5-435f-a271-33b25993a459 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:13.774033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7345c-84e5-435f-a271-33b25993a459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.808322 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fdf7345c-84e5-435f-a271-33b25993a459 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:13.809646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fdf7345c-84e5-435f-a271-33b25993a459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:13.906754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba0d2f-d5df-4246-b517-dd2e6dbf9072 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:13.907494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 92/181] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:13.908791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5846fe9-de50-4a04-9f29-093f64fce514 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:13.909080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5846fe9-de50-4a04-9f29-093f64fce514 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:13.909359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5846fe9-de50-4a04-9f29-093f64fce514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.930758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d5846fe9-de50-4a04-9f29-093f64fce514 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:13.932109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5846fe9-de50-4a04-9f29-093f64fce514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:13.956731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f9700-5522-4b45-a4a2-b95f7fdc3142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:13.957380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f9700-5522-4b45-a4a2-b95f7fdc3142 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:13.957662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f9700-5522-4b45-a4a2-b95f7fdc3142 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:13.957936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f9700-5522-4b45-a4a2-b95f7fdc3142 None admin] PATH_INFO: `/v3/users/60b0d29f45f6410f8f4e1856e03bc22b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:13.960674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f9700-5522-4b45-a4a2-b95f7fdc3142 None admin] RBAC: Authorizing `identity:get_user(user_id=60b0d29f45f6410f8f4e1856e03bc22b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:13.962389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f9700-5522-4b45-a4a2-b95f7fdc3142 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:13.964104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 90/182] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:13 2022] GET /identity/v3/users/60b0d29f45f6410f8f4e1856e03bc22b => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:13.965836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f9700-5522-4b45-a4a2-b95f7fdc3142 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:14.022826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 93/183] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:14.023588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5846fe9-de50-4a04-9f29-093f64fce514 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:14.045148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:14.050016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:14.093540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f9700-5522-4b45-a4a2-b95f7fdc3142 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:14.129151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c05505-03e4-49f7-83fa-f5f595d0e8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:14.129791 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c05505-03e4-49f7-83fa-f5f595d0e8bd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:14.130060 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c05505-03e4-49f7-83fa-f5f595d0e8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:14.130331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c05505-03e4-49f7-83fa-f5f595d0e8bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:14.134067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5846fe9-de50-4a04-9f29-093f64fce514 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:14.142513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c05505-03e4-49f7-83fa-f5f595d0e8bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-06c05505-03e4-49f7-83fa-f5f595d0e8bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:14.149367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:14.150219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 91/184] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:23:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:14.156400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c05505-03e4-49f7-83fa-f5f595d0e8bd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:14.179266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06bc72-2e99-4450-a3b4-48bed68aaf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:14.180177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06bc72-2e99-4450-a3b4-48bed68aaf6a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:14.180467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06bc72-2e99-4450-a3b4-48bed68aaf6a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:14.180746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06bc72-2e99-4450-a3b4-48bed68aaf6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:14.192269 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06bc72-2e99-4450-a3b4-48bed68aaf6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-aa06bc72-2e99-4450-a3b4-48bed68aaf6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:14.198618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:14.199492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:14.200310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 94/185] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:14.204272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06bc72-2e99-4450-a3b4-48bed68aaf6a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:14.261510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c05505-03e4-49f7-83fa-f5f595d0e8bd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:14.288714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:14.292420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:14.300013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2f15b-15b4-4a38-9ef1-79122c230db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:14.300655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2f15b-15b4-4a38-9ef1-79122c230db0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:14.302888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2f15b-15b4-4a38-9ef1-79122c230db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:14.302888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2f15b-15b4-4a38-9ef1-79122c230db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:14.311515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06bc72-2e99-4450-a3b4-48bed68aaf6a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:14.346092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163e21d-8235-4eff-b57c-bf426226e02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:14.346731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e21d-8235-4eff-b57c-bf426226e02b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:14.347037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e21d-8235-4eff-b57c-bf426226e02b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:14.347695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e21d-8235-4eff-b57c-bf426226e02b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:14.348882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163e21d-8235-4eff-b57c-bf426226e02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:14.350843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163e21d-8235-4eff-b57c-bf426226e02b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:14.352776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 95/186] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:14.353530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163e21d-8235-4eff-b57c-bf426226e02b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:14.376903 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2f15b-15b4-4a38-9ef1-79122c230db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:14.378386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2f15b-15b4-4a38-9ef1-79122c230db0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:14.463648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163e21d-8235-4eff-b57c-bf426226e02b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:14.478974 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 92/187] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:14.479814 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2f15b-15b4-4a38-9ef1-79122c230db0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:14.504281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9afa3c-d4ea-41e6-8b01-ae7bbfa0dbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:14.504948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9afa3c-d4ea-41e6-8b01-ae7bbfa0dbd1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:14.505084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:14.505230 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9afa3c-d4ea-41e6-8b01-ae7bbfa0dbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:14.505510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9afa3c-d4ea-41e6-8b01-ae7bbfa0dbd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:14.506761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9afa3c-d4ea-41e6-8b01-ae7bbfa0dbd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:14.507066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9afa3c-d4ea-41e6-8b01-ae7bbfa0dbd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:14.508507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9afa3c-d4ea-41e6-8b01-ae7bbfa0dbd1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:14.517504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 96/188] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:14.518544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9afa3c-d4ea-41e6-8b01-ae7bbfa0dbd1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:14.531460 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:14.592571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2f15b-15b4-4a38-9ef1-79122c230db0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:14.621410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9afa3c-d4ea-41e6-8b01-ae7bbfa0dbd1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:14.627862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8847415-9843-4526-b2b1-09599a68e3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:14.628509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8847415-9843-4526-b2b1-09599a68e3aa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:14.628797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8847415-9843-4526-b2b1-09599a68e3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:14.629072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8847415-9843-4526-b2b1-09599a68e3aa None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:14.629794 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8847415-9843-4526-b2b1-09599a68e3aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:14.630101 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8847415-9843-4526-b2b1-09599a68e3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:14.632273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8847415-9843-4526-b2b1-09599a68e3aa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:14.640375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 93/189] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:14.641000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8847415-9843-4526-b2b1-09599a68e3aa None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:14.659662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7e507-162f-405a-8b98-844858d0b36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:14.660670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7e507-162f-405a-8b98-844858d0b36a None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:14.661130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7e507-162f-405a-8b98-844858d0b36a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:14.661596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7e507-162f-405a-8b98-844858d0b36a None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:14.663279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7e507-162f-405a-8b98-844858d0b36a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:14.665145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7e507-162f-405a-8b98-844858d0b36a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:14.730323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:14.745118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8847415-9843-4526-b2b1-09599a68e3aa None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:14.752649 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 97/190] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:23:14 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:23:14.761415 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7e507-162f-405a-8b98-844858d0b36a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:14.791279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98315e3c-b1ac-4afc-a224-d3c664e952ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:14.791969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98315e3c-b1ac-4afc-a224-d3c664e952ce None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:14.792402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98315e3c-b1ac-4afc-a224-d3c664e952ce None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:14.792684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98315e3c-b1ac-4afc-a224-d3c664e952ce None admin] PATH_INFO: `/v3/projects/6c34cca62b0d4a439cd929a3b9f9de41` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:14.794064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98315e3c-b1ac-4afc-a224-d3c664e952ce None admin] RBAC: Authorizing `identity:get_project(project_id=6c34cca62b0d4a439cd929a3b9f9de41)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:14.795532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98315e3c-b1ac-4afc-a224-d3c664e952ce None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:14.797577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 94/191] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:14 2022] GET /identity/v3/projects/6c34cca62b0d4a439cd929a3b9f9de41 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:14.804057 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98315e3c-b1ac-4afc-a224-d3c664e952ce None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:14.852238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:14.892321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7e507-162f-405a-8b98-844858d0b36a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:14.923159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98315e3c-b1ac-4afc-a224-d3c664e952ce None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:14.936983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078769f6-3a08-43a7-93ed-0d4e9bda6882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:14.937643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078769f6-3a08-43a7-93ed-0d4e9bda6882 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:14.937931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078769f6-3a08-43a7-93ed-0d4e9bda6882 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:14.938210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078769f6-3a08-43a7-93ed-0d4e9bda6882 None admin] PATH_INFO: `/v3/users/8f37dff953764cfdb5bea1432e35ba87` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:14.956334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078769f6-3a08-43a7-93ed-0d4e9bda6882 None admin] RBAC: Authorizing `identity:get_user(user_id=8f37dff953764cfdb5bea1432e35ba87)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:14.957782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078769f6-3a08-43a7-93ed-0d4e9bda6882 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:14.959579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 98/192] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:14 2022] GET /identity/v3/users/8f37dff953764cfdb5bea1432e35ba87 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:14.964047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078769f6-3a08-43a7-93ed-0d4e9bda6882 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:14.965497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481632df-e7ef-4a66-b2d3-b6d0dafd8c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:14.967207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481632df-e7ef-4a66-b2d3-b6d0dafd8c5c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:14.967506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481632df-e7ef-4a66-b2d3-b6d0dafd8c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:14.967776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481632df-e7ef-4a66-b2d3-b6d0dafd8c5c None admin] PATH_INFO: `/v3/users/8f37dff953764cfdb5bea1432e35ba87` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:14.969031 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481632df-e7ef-4a66-b2d3-b6d0dafd8c5c None admin] RBAC: Authorizing `identity:get_user(user_id=8f37dff953764cfdb5bea1432e35ba87)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:14.970490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481632df-e7ef-4a66-b2d3-b6d0dafd8c5c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:14.972777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 95/193] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:14 2022] GET /identity/v3/users/8f37dff953764cfdb5bea1432e35ba87 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:14.978558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86c410-166b-4059-a712-db6af2ea9b78 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:14.978851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86c410-166b-4059-a712-db6af2ea9b78 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:14.979128 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86c410-166b-4059-a712-db6af2ea9b78 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:14.980492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 96/194] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:14.981132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86c410-166b-4059-a712-db6af2ea9b78 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:15.013414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:15.095599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078769f6-3a08-43a7-93ed-0d4e9bda6882 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:15.119196 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86c410-166b-4059-a712-db6af2ea9b78 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:15.127278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:15.139442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00fd48-3f05-494e-810f-cdc4da558880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:15.144173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00fd48-3f05-494e-810f-cdc4da558880 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:15.144173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00fd48-3f05-494e-810f-cdc4da558880 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:15.144173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00fd48-3f05-494e-810f-cdc4da558880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:15.144173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00fd48-3f05-494e-810f-cdc4da558880 None admin] RBAC: Adding query filter params (role.id=525e499f44594268b4f362c62bd35e00, scope.project.id=6c34cca62b0d4a439cd929a3b9f9de41, user.id=60b0d29f45f6410f8f4e1856e03bc22b) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:15.144173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00fd48-3f05-494e-810f-cdc4da558880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:15.144173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00fd48-3f05-494e-810f-cdc4da558880 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:15.149681 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 99/195] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:14 2022] GET /identity/v3/role_assignments?user.id=60b0d29f45f6410f8f4e1856e03bc22b&scope.project.id=6c34cca62b0d4a439cd929a3b9f9de41&role.id=525e499f44594268b4f362c62bd35e00 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:15.150420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00fd48-3f05-494e-810f-cdc4da558880 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:15.161702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8497f81-e76e-4613-8267-23990bc579ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:15.163374 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8497f81-e76e-4613-8267-23990bc579ee None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:15.168006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8497f81-e76e-4613-8267-23990bc579ee None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:15.168006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8497f81-e76e-4613-8267-23990bc579ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:15.261995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8497f81-e76e-4613-8267-23990bc579ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:15.264036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8497f81-e76e-4613-8267-23990bc579ee None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:15.269390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00fd48-3f05-494e-810f-cdc4da558880 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:15.292240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:15.310948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b6408-0855-4b9b-b649-cbc33bc22357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:15.312062 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b6408-0855-4b9b-b649-cbc33bc22357 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:15.312194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b6408-0855-4b9b-b649-cbc33bc22357 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:15.312647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b6408-0855-4b9b-b649-cbc33bc22357 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:15.416824 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:15.422775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 97/196] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:15.428754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112562d-f739-4493-bfab-13192c916dc9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:15.429943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112562d-f739-4493-bfab-13192c916dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:15.430324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112562d-f739-4493-bfab-13192c916dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:15.441208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b6408-0855-4b9b-b649-cbc33bc22357 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:15.441944 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b6408-0855-4b9b-b649-cbc33bc22357 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:15.460568 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5112562d-f739-4493-bfab-13192c916dc9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:15.462083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5112562d-f739-4493-bfab-13192c916dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:15.541038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 98/197] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:15.542277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112562d-f739-4493-bfab-13192c916dc9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:15.567415 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 100/198] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:15.573861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b6408-0855-4b9b-b649-cbc33bc22357 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:15.596879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:15.651574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112562d-f739-4493-bfab-13192c916dc9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:15.687605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:15.693597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940990f-a4e8-4664-a819-df68f2421889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:15.695415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940990f-a4e8-4664-a819-df68f2421889 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:15.695901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940990f-a4e8-4664-a819-df68f2421889 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:15.696394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940990f-a4e8-4664-a819-df68f2421889 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:15.697944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940990f-a4e8-4664-a819-df68f2421889 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:15.699669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940990f-a4e8-4664-a819-df68f2421889 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:15.701994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 99/199] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:15 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:15.702941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940990f-a4e8-4664-a819-df68f2421889 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:15.708023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b6408-0855-4b9b-b649-cbc33bc22357 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:15.744890 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42622b-2c7b-4095-b26d-d9f055f4ee4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:15.745545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42622b-2c7b-4095-b26d-d9f055f4ee4d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:15.745847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42622b-2c7b-4095-b26d-d9f055f4ee4d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:15.746825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42622b-2c7b-4095-b26d-d9f055f4ee4d None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:15.747526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42622b-2c7b-4095-b26d-d9f055f4ee4d None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:15.749108 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42622b-2c7b-4095-b26d-d9f055f4ee4d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:15.750851 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 101/200] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:15 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:15.751738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42622b-2c7b-4095-b26d-d9f055f4ee4d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:15.820754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940990f-a4e8-4664-a819-df68f2421889 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:15.843458 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:15.862559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c607ada-17cb-4b75-8b66-9d727188afe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:15.870246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c607ada-17cb-4b75-8b66-9d727188afe1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:15.870246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c607ada-17cb-4b75-8b66-9d727188afe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:15.870246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c607ada-17cb-4b75-8b66-9d727188afe1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:15.876807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c607ada-17cb-4b75-8b66-9d727188afe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:15.878063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42622b-2c7b-4095-b26d-d9f055f4ee4d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2c607ada-17cb-4b75-8b66-9d727188afe1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:15.883764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:15.884790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:15.885788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:15.888496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 100/201] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:15.889420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c607ada-17cb-4b75-8b66-9d727188afe1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:15.929398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491cdba-a2e4-4fb4-8322-5115505eaf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:15.930086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491cdba-a2e4-4fb4-8322-5115505eaf32 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:15.931290 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491cdba-a2e4-4fb4-8322-5115505eaf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:15.931640 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491cdba-a2e4-4fb4-8322-5115505eaf32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:15.932535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491cdba-a2e4-4fb4-8322-5115505eaf32 None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=6a913e1d666748509269614e0c708389, user.id=8f37dff953764cfdb5bea1432e35ba87) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:15.932914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491cdba-a2e4-4fb4-8322-5115505eaf32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:15.934418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491cdba-a2e4-4fb4-8322-5115505eaf32 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:15.948045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 102/202] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:15 2022] GET /identity/v3/role_assignments?user.id=8f37dff953764cfdb5bea1432e35ba87&scope.project.id=6a913e1d666748509269614e0c708389&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:15.948045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491cdba-a2e4-4fb4-8322-5115505eaf32 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:15.978805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:16.074466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c607ada-17cb-4b75-8b66-9d727188afe1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:16.079073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491cdba-a2e4-4fb4-8322-5115505eaf32 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:16.127093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de878e7b-22c3-4121-bffa-d2a0368daf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:16.127730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de878e7b-22c3-4121-bffa-d2a0368daf80 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:16.128029 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de878e7b-22c3-4121-bffa-d2a0368daf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:16.128314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de878e7b-22c3-4121-bffa-d2a0368daf80 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:16.129049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de878e7b-22c3-4121-bffa-d2a0368daf80 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:16.129357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de878e7b-22c3-4121-bffa-d2a0368daf80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:16.130766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de878e7b-22c3-4121-bffa-d2a0368daf80 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:16.150437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 103/203] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:16.151663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af3da6-c789-4760-84ef-a9cff747dcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:16.152331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af3da6-c789-4760-84ef-a9cff747dcb0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:16.152620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af3da6-c789-4760-84ef-a9cff747dcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:16.152900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af3da6-c789-4760-84ef-a9cff747dcb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:16.153716 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af3da6-c789-4760-84ef-a9cff747dcb0 None admin] RBAC: Adding query filter params (role.id=e3c5a858a45c41db8ff3e883788d75bc, scope.project.id=6a913e1d666748509269614e0c708389, user.id=8f37dff953764cfdb5bea1432e35ba87) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:16.154030 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af3da6-c789-4760-84ef-a9cff747dcb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:16.155907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de878e7b-22c3-4121-bffa-d2a0368daf80 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:16.157171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af3da6-c789-4760-84ef-a9cff747dcb0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:16.178322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 101/204] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:15 2022] GET /identity/v3/role_assignments?user.id=8f37dff953764cfdb5bea1432e35ba87&scope.project.id=6a913e1d666748509269614e0c708389&role.id=e3c5a858a45c41db8ff3e883788d75bc => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:16.255292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:16.271295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de878e7b-22c3-4121-bffa-d2a0368daf80 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:16.312014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34516464-7f2d-47bf-8800-9fe78cf29ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:16.312681 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34516464-7f2d-47bf-8800-9fe78cf29ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:16.312965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34516464-7f2d-47bf-8800-9fe78cf29ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:16.313241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34516464-7f2d-47bf-8800-9fe78cf29ea4 None admin] PATH_INFO: `/v3/users/ed0dacdea889430f86b9cda4063b3ba9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:16.314455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34516464-7f2d-47bf-8800-9fe78cf29ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0dacdea889430f86b9cda4063b3ba9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:16.315915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34516464-7f2d-47bf-8800-9fe78cf29ea4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:16.317794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 104/205] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:16 2022] GET /identity/v3/users/ed0dacdea889430f86b9cda4063b3ba9 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:16.323508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af3da6-c789-4760-84ef-a9cff747dcb0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:16.374764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:16.434894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af3da6-c789-4760-84ef-a9cff747dcb0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:16.469088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492dc33-6eb6-45aa-9238-2409b4030e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:16.469710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492dc33-6eb6-45aa-9238-2409b4030e96 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:16.469994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492dc33-6eb6-45aa-9238-2409b4030e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:16.470273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492dc33-6eb6-45aa-9238-2409b4030e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:16.540521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492dc33-6eb6-45aa-9238-2409b4030e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:16.541983 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492dc33-6eb6-45aa-9238-2409b4030e96 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:16.584898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:16.643510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 102/206] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 07:23:16.650003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34516464-7f2d-47bf-8800-9fe78cf29ea4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:16.753866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34516464-7f2d-47bf-8800-9fe78cf29ea4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:16.793622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470366e3-2128-430d-8852-d6021a477603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:16.794280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470366e3-2128-430d-8852-d6021a477603 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:16.794570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470366e3-2128-430d-8852-d6021a477603 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:16.794846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470366e3-2128-430d-8852-d6021a477603 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:16.796061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470366e3-2128-430d-8852-d6021a477603 None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:16.797514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470366e3-2128-430d-8852-d6021a477603 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:16.799338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 105/207] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:16 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:16.805222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492dc33-6eb6-45aa-9238-2409b4030e96 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:16.907935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492dc33-6eb6-45aa-9238-2409b4030e96 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:16.944473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b12667-974c-4bf1-947d-68375f4c770d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:16.945148 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b12667-974c-4bf1-947d-68375f4c770d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:16.945431 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b12667-974c-4bf1-947d-68375f4c770d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:16.945708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b12667-974c-4bf1-947d-68375f4c770d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:16.946507 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b12667-974c-4bf1-947d-68375f4c770d None admin] RBAC: Adding query filter params (role.id=e2331f0635214139bdac452081a3f310, scope.project.id=ce53e46475f341a2af8bf9da4700fffe, user.id=ed0dacdea889430f86b9cda4063b3ba9) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:16.946816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b12667-974c-4bf1-947d-68375f4c770d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:16.949867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b12667-974c-4bf1-947d-68375f4c770d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:16.958454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 103/208] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:16 2022] GET /identity/v3/role_assignments?user.id=ed0dacdea889430f86b9cda4063b3ba9&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=e2331f0635214139bdac452081a3f310 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:17.156567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73b7e8-5c0c-4980-9e92-4ac95f9459c0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:17.157060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73b7e8-5c0c-4980-9e92-4ac95f9459c0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:17.157158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73b7e8-5c0c-4980-9e92-4ac95f9459c0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:17.158817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 106/209] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:17.164644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c63484-8b23-4a6e-82da-4564a4eedfdd None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:17.164938 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c63484-8b23-4a6e-82da-4564a4eedfdd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:17.165219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c63484-8b23-4a6e-82da-4564a4eedfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:17.179388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:17.185820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-34c63484-8b23-4a6e-82da-4564a4eedfdd None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:17.187023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-34c63484-8b23-4a6e-82da-4564a4eedfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:17.262137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 104/210] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:17.286565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73b7e8-5c0c-4980-9e92-4ac95f9459c0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:17.308226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:17.405439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73b7e8-5c0c-4980-9e92-4ac95f9459c0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:17.446131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ada9c9-c271-42df-b142-520cda563a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:17.446775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada9c9-c271-42df-b142-520cda563a36 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:17.447059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada9c9-c271-42df-b142-520cda563a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:17.447339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada9c9-c271-42df-b142-520cda563a36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:17.457936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ada9c9-c271-42df-b142-520cda563a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b2ada9c9-c271-42df-b142-520cda563a36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:17.463030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:17.463883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:17.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 107/211] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:17.471437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c63484-8b23-4a6e-82da-4564a4eedfdd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:17.578126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c63484-8b23-4a6e-82da-4564a4eedfdd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:17.613912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a28e1-98c7-4bfc-9ec6-253b3ea0e59f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:17.614207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a28e1-98c7-4bfc-9ec6-253b3ea0e59f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:17.614477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a28e1-98c7-4bfc-9ec6-253b3ea0e59f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:17.616800 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 108/212] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:17.617898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d5dca-a955-44fd-a112-7cfc33cdf6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:17.618770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d5dca-a955-44fd-a112-7cfc33cdf6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:17.619246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d5dca-a955-44fd-a112-7cfc33cdf6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:17.619723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d5dca-a955-44fd-a112-7cfc33cdf6a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:17.620922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d5dca-a955-44fd-a112-7cfc33cdf6a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:17.621443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d5dca-a955-44fd-a112-7cfc33cdf6a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:17.621681 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633f194-afbf-4f21-9938-349134c9f7d7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:17.622682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633f194-afbf-4f21-9938-349134c9f7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:17.622969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633f194-afbf-4f21-9938-349134c9f7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:17.623628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d5dca-a955-44fd-a112-7cfc33cdf6a6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:17.626323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:17.633599 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 105/213] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:17.641102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d5dca-a955-44fd-a112-7cfc33cdf6a6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:17.644177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9633f194-afbf-4f21-9938-349134c9f7d7 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:17.645548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9633f194-afbf-4f21-9938-349134c9f7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:17.737810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 109/214] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:17.762390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633f194-afbf-4f21-9938-349134c9f7d7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:17.773634 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:17.779337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d5dca-a955-44fd-a112-7cfc33cdf6a6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:17.820348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4ce5a-8d3d-44cf-b545-46c1f6a16f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:17.821215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4ce5a-8d3d-44cf-b545-46c1f6a16f7a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:17.821540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4ce5a-8d3d-44cf-b545-46c1f6a16f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:17.821921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4ce5a-8d3d-44cf-b545-46c1f6a16f7a None admin] PATH_INFO: `/v3/users/b51a13f6fd5f429b80273d468335a1d8` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:17.837680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4ce5a-8d3d-44cf-b545-46c1f6a16f7a None admin] RBAC: Authorizing `identity:get_user(user_id=b51a13f6fd5f429b80273d468335a1d8)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:17.839604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4ce5a-8d3d-44cf-b545-46c1f6a16f7a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:17.841597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 106/215] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:17 2022] GET /identity/v3/users/b51a13f6fd5f429b80273d468335a1d8 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:17.847007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4ce5a-8d3d-44cf-b545-46c1f6a16f7a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:17.889598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633f194-afbf-4f21-9938-349134c9f7d7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:17.913454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:17.931047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233642ae-aa16-4bef-80f0-febfb0db7e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:17.932013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233642ae-aa16-4bef-80f0-febfb0db7e20 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:17.940537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233642ae-aa16-4bef-80f0-febfb0db7e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:17.940537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233642ae-aa16-4bef-80f0-febfb0db7e20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:17.940537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233642ae-aa16-4bef-80f0-febfb0db7e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:17.940537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233642ae-aa16-4bef-80f0-febfb0db7e20 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:17.940987 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 110/216] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:17.942877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233642ae-aa16-4bef-80f0-febfb0db7e20 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:17.968946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4ce5a-8d3d-44cf-b545-46c1f6a16f7a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:18.016685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e2455-4464-448f-a0d9-5cf0e3e2f8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:18.017534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e2455-4464-448f-a0d9-5cf0e3e2f8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:18.018077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e2455-4464-448f-a0d9-5cf0e3e2f8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:18.018513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e2455-4464-448f-a0d9-5cf0e3e2f8e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:18.031933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:18.064067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233642ae-aa16-4bef-80f0-febfb0db7e20 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:18.100238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e2455-4464-448f-a0d9-5cf0e3e2f8e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:18.101786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e2455-4464-448f-a0d9-5cf0e3e2f8e7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:18.115829 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881281ce-2c91-4feb-b5af-a2d81fcebe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:18.116798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881281ce-2c91-4feb-b5af-a2d81fcebe68 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:18.117230 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881281ce-2c91-4feb-b5af-a2d81fcebe68 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:18.117655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881281ce-2c91-4feb-b5af-a2d81fcebe68 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:18.119242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881281ce-2c91-4feb-b5af-a2d81fcebe68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:18.121189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881281ce-2c91-4feb-b5af-a2d81fcebe68 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:18.165454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:18.214203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 107/217] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 07:23:18.219773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e2455-4464-448f-a0d9-5cf0e3e2f8e7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:18.225546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 111/218] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:23:17 2022] POST /identity/v3/users => generated 297 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:23:18.345866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e2455-4464-448f-a0d9-5cf0e3e2f8e7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:18.358435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe13782-f70f-4196-92e6-ca873848c2f0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:18.359059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe13782-f70f-4196-92e6-ca873848c2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:18.359567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe13782-f70f-4196-92e6-ca873848c2f0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:18.361424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 112/219] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:18.367301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009a633-0676-4aa6-b31f-b88da1fd3fb5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:18.368411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009a633-0676-4aa6-b31f-b88da1fd3fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:18.368411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009a633-0676-4aa6-b31f-b88da1fd3fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:18.390072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31267ba-015e-4508-a6d7-7885d967275c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:18.390277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6009a633-0676-4aa6-b31f-b88da1fd3fb5 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:18.390718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31267ba-015e-4508-a6d7-7885d967275c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:18.391000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31267ba-015e-4508-a6d7-7885d967275c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:18.391276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31267ba-015e-4508-a6d7-7885d967275c None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:18.391636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6009a633-0676-4aa6-b31f-b88da1fd3fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:18.392509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31267ba-015e-4508-a6d7-7885d967275c None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:18.393951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31267ba-015e-4508-a6d7-7885d967275c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:18.395802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 108/220] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:18 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:18.402027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31267ba-015e-4508-a6d7-7885d967275c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:18.432068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:18.484088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 113/221] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:18.507519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009a633-0676-4aa6-b31f-b88da1fd3fb5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:18.522333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31267ba-015e-4508-a6d7-7885d967275c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:18.561604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceab632-a2d0-4174-9922-eebbb09adb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:18.561604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceab632-a2d0-4174-9922-eebbb09adb39 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:18.561907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceab632-a2d0-4174-9922-eebbb09adb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:18.562102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceab632-a2d0-4174-9922-eebbb09adb39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:18.562904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceab632-a2d0-4174-9922-eebbb09adb39 None admin] RBAC: Adding query filter params (role.id=525e499f44594268b4f362c62bd35e00, scope.project.id=6a913e1d666748509269614e0c708389, user.id=b51a13f6fd5f429b80273d468335a1d8) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:18.563384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceab632-a2d0-4174-9922-eebbb09adb39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:18.564681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceab632-a2d0-4174-9922-eebbb09adb39 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:18.571326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 109/222] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:18 2022] GET /identity/v3/role_assignments?user.id=b51a13f6fd5f429b80273d468335a1d8&scope.project.id=6a913e1d666748509269614e0c708389&role.id=525e499f44594268b4f362c62bd35e00 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:18.588126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:18.622561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d655ed-90ee-4da0-8e0f-79662c9b9cf2 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:18.622842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d655ed-90ee-4da0-8e0f-79662c9b9cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:18.623033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d655ed-90ee-4da0-8e0f-79662c9b9cf2 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:18.624580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 110/223] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:23:18.630504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9f458-e177-4ead-85cd-ca7c05b53e0e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:18.630857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9f458-e177-4ead-85cd-ca7c05b53e0e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:18.631709 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9f458-e177-4ead-85cd-ca7c05b53e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:18.640608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009a633-0676-4aa6-b31f-b88da1fd3fb5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:18.655265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-21f9f458-e177-4ead-85cd-ca7c05b53e0e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:18.656543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-21f9f458-e177-4ead-85cd-ca7c05b53e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:18.689652 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc66b88-f39b-459a-a3db-8f307130d4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:18.690775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc66b88-f39b-459a-a3db-8f307130d4fc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:18.691332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc66b88-f39b-459a-a3db-8f307130d4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:18.691623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc66b88-f39b-459a-a3db-8f307130d4fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:18.708537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc66b88-f39b-459a-a3db-8f307130d4fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5fc66b88-f39b-459a-a3db-8f307130d4fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:18.716136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:18.717071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:18.717877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:18.718816 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 114/224] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:23:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:18.725287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc66b88-f39b-459a-a3db-8f307130d4fc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:18.750975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 111/225] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:18.775053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9f458-e177-4ead-85cd-ca7c05b53e0e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:18.798675 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:18.839910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc66b88-f39b-459a-a3db-8f307130d4fc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:18.865755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:18.878744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aabc23-057f-4b3f-8d86-6f15db59da2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:18.879655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aabc23-057f-4b3f-8d86-6f15db59da2c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:18.880154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aabc23-057f-4b3f-8d86-6f15db59da2c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:18.880596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aabc23-057f-4b3f-8d86-6f15db59da2c None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:18.882622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aabc23-057f-4b3f-8d86-6f15db59da2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:18.883140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aabc23-057f-4b3f-8d86-6f15db59da2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:18.884766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aabc23-057f-4b3f-8d86-6f15db59da2c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:18.893883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 115/226] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:18.895922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9f458-e177-4ead-85cd-ca7c05b53e0e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:18.901013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aabc23-057f-4b3f-8d86-6f15db59da2c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:18.932060 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f826eb8-084b-425c-8ff0-8289097a31cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:18.932715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f826eb8-084b-425c-8ff0-8289097a31cf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:18.932994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f826eb8-084b-425c-8ff0-8289097a31cf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:18.933273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f826eb8-084b-425c-8ff0-8289097a31cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:18.944085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f826eb8-084b-425c-8ff0-8289097a31cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7f826eb8-084b-425c-8ff0-8289097a31cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:18.949402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:18.950328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:18.951127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 112/227] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:18.963601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f826eb8-084b-425c-8ff0-8289097a31cf None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:19.082139 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aabc23-057f-4b3f-8d86-6f15db59da2c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:19.093002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:19.133564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f826eb8-084b-425c-8ff0-8289097a31cf None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:19.146746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c463f-f57b-4d58-91ca-3e59f1895722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:19.147261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c463f-f57b-4d58-91ca-3e59f1895722 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:19.147546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c463f-f57b-4d58-91ca-3e59f1895722 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:19.147826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c463f-f57b-4d58-91ca-3e59f1895722 None admin] PATH_INFO: `/v3/users/8f37dff953764cfdb5bea1432e35ba87` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:19.149153 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c463f-f57b-4d58-91ca-3e59f1895722 None admin] RBAC: Authorizing `identity:get_user(user_id=8f37dff953764cfdb5bea1432e35ba87)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:19.150777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c463f-f57b-4d58-91ca-3e59f1895722 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:19.152886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 116/228] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:18 2022] GET /identity/v3/users/8f37dff953764cfdb5bea1432e35ba87 => generated 283 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:19.158020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c463f-f57b-4d58-91ca-3e59f1895722 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:19.172960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a007369-3248-43fd-9c81-848502d84b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:19.173580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a007369-3248-43fd-9c81-848502d84b05 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:19.173860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a007369-3248-43fd-9c81-848502d84b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:19.174133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a007369-3248-43fd-9c81-848502d84b05 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:19.174855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a007369-3248-43fd-9c81-848502d84b05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:19.175158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a007369-3248-43fd-9c81-848502d84b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:19.176552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a007369-3248-43fd-9c81-848502d84b05 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:19.191219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 113/229] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:19.200939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a007369-3248-43fd-9c81-848502d84b05 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:19.220372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:19.264469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c463f-f57b-4d58-91ca-3e59f1895722 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:19.301197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a480e9-d935-478c-b65c-2c0d848669ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:19.301940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a480e9-d935-478c-b65c-2c0d848669ff None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:19.302230 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a480e9-d935-478c-b65c-2c0d848669ff None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:19.302506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a480e9-d935-478c-b65c-2c0d848669ff None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:19.304423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a480e9-d935-478c-b65c-2c0d848669ff None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:19.306147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a480e9-d935-478c-b65c-2c0d848669ff None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:19.308225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 117/230] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:19 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:19.313381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a480e9-d935-478c-b65c-2c0d848669ff None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:19.423115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a480e9-d935-478c-b65c-2c0d848669ff None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:19.460458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:19.462741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd4c07-e1c4-49fd-a9ab-45a743ec50c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:19.463461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd4c07-e1c4-49fd-a9ab-45a743ec50c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:19.463847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd4c07-e1c4-49fd-a9ab-45a743ec50c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:19.464308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd4c07-e1c4-49fd-a9ab-45a743ec50c2 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389/users/8f37dff953764cfdb5bea1432e35ba87/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:19.466774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd4c07-e1c4-49fd-a9ab-45a743ec50c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a913e1d666748509269614e0c708389, user_id=8f37dff953764cfdb5bea1432e35ba87, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:19.468443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd4c07-e1c4-49fd-a9ab-45a743ec50c2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:19.474404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a007369-3248-43fd-9c81-848502d84b05 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:19.518742 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31842e2d-1659-4db5-aeac-d69e25086e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:19.519476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31842e2d-1659-4db5-aeac-d69e25086e80 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:19.519767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31842e2d-1659-4db5-aeac-d69e25086e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:19.520112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31842e2d-1659-4db5-aeac-d69e25086e80 None admin] PATH_INFO: `/v3/users/8f37dff953764cfdb5bea1432e35ba87` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:19.521455 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31842e2d-1659-4db5-aeac-d69e25086e80 None admin] RBAC: Authorizing `identity:get_user(user_id=8f37dff953764cfdb5bea1432e35ba87)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:19.523354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31842e2d-1659-4db5-aeac-d69e25086e80 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:19.525377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 114/231] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:19 2022] GET /identity/v3/users/8f37dff953764cfdb5bea1432e35ba87 => generated 283 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:19.530900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31842e2d-1659-4db5-aeac-d69e25086e80 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:19.546922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:19.564872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 118/232] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:23:19 2022] PUT /identity/v3/projects/6a913e1d666748509269614e0c708389/users/8f37dff953764cfdb5bea1432e35ba87/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:23:19.657059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31842e2d-1659-4db5-aeac-d69e25086e80 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:19.698465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11670bc9-c753-44e4-8694-f8e183e998fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:19.699020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11670bc9-c753-44e4-8694-f8e183e998fa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:19.699302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11670bc9-c753-44e4-8694-f8e183e998fa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:19.699576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11670bc9-c753-44e4-8694-f8e183e998fa None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:19.700814 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11670bc9-c753-44e4-8694-f8e183e998fa None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:19.702334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11670bc9-c753-44e4-8694-f8e183e998fa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:19.708159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 115/233] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:19 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:19.711081 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd4c07-e1c4-49fd-a9ab-45a743ec50c2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:19.843605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd4c07-e1c4-49fd-a9ab-45a743ec50c2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:19.885232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2d5e4-55cb-4842-b2d9-0a1d58cbd3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:19.886210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2d5e4-55cb-4842-b2d9-0a1d58cbd3d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:19.886676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2d5e4-55cb-4842-b2d9-0a1d58cbd3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:19.887130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2d5e4-55cb-4842-b2d9-0a1d58cbd3d2 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389/users/8f37dff953764cfdb5bea1432e35ba87/roles/e3c5a858a45c41db8ff3e883788d75bc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:19.889581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2d5e4-55cb-4842-b2d9-0a1d58cbd3d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a913e1d666748509269614e0c708389, user_id=8f37dff953764cfdb5bea1432e35ba87, role_id=e3c5a858a45c41db8ff3e883788d75bc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:19.891233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2d5e4-55cb-4842-b2d9-0a1d58cbd3d2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:20.011168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 119/234] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:23:19 2022] PUT /identity/v3/projects/6a913e1d666748509269614e0c708389/users/8f37dff953764cfdb5bea1432e35ba87/roles/e3c5a858a45c41db8ff3e883788d75bc => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:20.971191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1fd59-9075-455d-a13a-8b69b57138a9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:20.972060 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1fd59-9075-455d-a13a-8b69b57138a9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:20.972552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1fd59-9075-455d-a13a-8b69b57138a9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:20.974401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 116/235] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:20.980338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bb39f-fe0c-4eca-b0aa-a9e951a5f6f5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:20.981186 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bb39f-fe0c-4eca-b0aa-a9e951a5f6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:20.981700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bb39f-fe0c-4eca-b0aa-a9e951a5f6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.002802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-373bb39f-fe0c-4eca-b0aa-a9e951a5f6f5 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:21.004401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-373bb39f-fe0c-4eca-b0aa-a9e951a5f6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:21.019605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0fc9b-503f-420c-a249-bd6612b8bf42 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.020246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0fc9b-503f-420c-a249-bd6612b8bf42 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.020751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0fc9b-503f-420c-a249-bd6612b8bf42 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.022847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 117/236] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:23:21.028616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56b19-d216-43e6-8a40-ed65d3c1ef99 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.029165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56b19-d216-43e6-8a40-ed65d3c1ef99 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.029667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56b19-d216-43e6-8a40-ed65d3c1ef99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.053728 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-84a56b19-d216-43e6-8a40-ed65d3c1ef99 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:21.055430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-84a56b19-d216-43e6-8a40-ed65d3c1ef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:21.086785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:21.126653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 120/237] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:21.128128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa367d-8117-4318-b4c4-81292bd16ca5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.128413 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa367d-8117-4318-b4c4-81292bd16ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.128685 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa367d-8117-4318-b4c4-81292bd16ca5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.129925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 121/238] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:21.141772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b79a7c-7640-4c94-9d6c-154fc447b68f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.142059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b79a7c-7640-4c94-9d6c-154fc447b68f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.142349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b79a7c-7640-4c94-9d6c-154fc447b68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.171279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-22b79a7c-7640-4c94-9d6c-154fc447b68f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:21.172615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-22b79a7c-7640-4c94-9d6c-154fc447b68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:21.173934 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 118/239] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:21.174950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a56b19-d216-43e6-8a40-ed65d3c1ef99 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:21.183357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:21.246519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 122/240] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:21.247633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b79a7c-7640-4c94-9d6c-154fc447b68f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:21.306516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a56b19-d216-43e6-8a40-ed65d3c1ef99 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:21.343546 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632988e-8535-48ad-8451-bfc23c3ccaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:21.344375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632988e-8535-48ad-8451-bfc23c3ccaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.344663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632988e-8535-48ad-8451-bfc23c3ccaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.344943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632988e-8535-48ad-8451-bfc23c3ccaf4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.352475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:21.355374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b79a7c-7640-4c94-9d6c-154fc447b68f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:21.357857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f632988e-8535-48ad-8451-bfc23c3ccaf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f632988e-8535-48ad-8451-bfc23c3ccaf4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:21.363314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:21.369155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 119/241] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:21.370049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632988e-8535-48ad-8451-bfc23c3ccaf4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:21.397384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d33e6f-4c35-4df4-988d-24f5a2c05082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:21.397799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d33e6f-4c35-4df4-988d-24f5a2c05082 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.398086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d33e6f-4c35-4df4-988d-24f5a2c05082 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.398365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d33e6f-4c35-4df4-988d-24f5a2c05082 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.399514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d33e6f-4c35-4df4-988d-24f5a2c05082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:21.401132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d33e6f-4c35-4df4-988d-24f5a2c05082 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:21.402951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 123/242] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:21.403632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d33e6f-4c35-4df4-988d-24f5a2c05082 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:21.443436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:21.481509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632988e-8535-48ad-8451-bfc23c3ccaf4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:21.511895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d33e6f-4c35-4df4-988d-24f5a2c05082 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:21.518797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea165dc-5a1a-4306-a8ae-49d0165790ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:21.519699 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea165dc-5a1a-4306-a8ae-49d0165790ed None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.520276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea165dc-5a1a-4306-a8ae-49d0165790ed None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.520727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea165dc-5a1a-4306-a8ae-49d0165790ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.533094 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea165dc-5a1a-4306-a8ae-49d0165790ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-aea165dc-5a1a-4306-a8ae-49d0165790ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:21.540042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:21.541012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:21.541801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:21.544108 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 120/243] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:21.549409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea165dc-5a1a-4306-a8ae-49d0165790ed None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:21.551197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbd05a-9201-4070-845e-b7350d2ffba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:21.551903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd05a-9201-4070-845e-b7350d2ffba1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.552226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd05a-9201-4070-845e-b7350d2ffba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.552548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd05a-9201-4070-845e-b7350d2ffba1 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.553333 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbd05a-9201-4070-845e-b7350d2ffba1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:21.553648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbd05a-9201-4070-845e-b7350d2ffba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:21.555519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbd05a-9201-4070-845e-b7350d2ffba1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:21.564850 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 124/244] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:21.570396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbd05a-9201-4070-845e-b7350d2ffba1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:21.605841 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:21.661116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea165dc-5a1a-4306-a8ae-49d0165790ed None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:21.682542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbd05a-9201-4070-845e-b7350d2ffba1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:21.690162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:21.700261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967108ec-8541-4c80-9b9b-6d145e1cba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:21.701066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967108ec-8541-4c80-9b9b-6d145e1cba51 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.701495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967108ec-8541-4c80-9b9b-6d145e1cba51 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.701915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967108ec-8541-4c80-9b9b-6d145e1cba51 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.702852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967108ec-8541-4c80-9b9b-6d145e1cba51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:21.704867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967108ec-8541-4c80-9b9b-6d145e1cba51 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:21.720927 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d32ed-425e-451b-960f-dcdb3ba80291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:21.721551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d32ed-425e-451b-960f-dcdb3ba80291 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.721834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d32ed-425e-451b-960f-dcdb3ba80291 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.722113 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d32ed-425e-451b-960f-dcdb3ba80291 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.722851 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d32ed-425e-451b-960f-dcdb3ba80291 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:21.723245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d32ed-425e-451b-960f-dcdb3ba80291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:21.724756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d32ed-425e-451b-960f-dcdb3ba80291 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:21.736148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 125/245] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:21.736148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d32ed-425e-451b-960f-dcdb3ba80291 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:21.789724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 121/246] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:23:21 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:23:21.790726 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967108ec-8541-4c80-9b9b-6d145e1cba51 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:21.865185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:21.879977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d32ed-425e-451b-960f-dcdb3ba80291 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:21.923739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967108ec-8541-4c80-9b9b-6d145e1cba51 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:21.967960 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9382c62-d4e9-4aaa-81a8-eb1d8111b89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:21.969022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382c62-d4e9-4aaa-81a8-eb1d8111b89a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.969870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a69a96-4a51-4098-8a0d-e2c106913d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:21.970399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382c62-d4e9-4aaa-81a8-eb1d8111b89a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.971259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a69a96-4a51-4098-8a0d-e2c106913d73 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:21.972101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382c62-d4e9-4aaa-81a8-eb1d8111b89a None admin] PATH_INFO: `/v3/users/a3f295c9d0b6422b9f50eac9eb3c9f15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.973282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a69a96-4a51-4098-8a0d-e2c106913d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:21.973771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a69a96-4a51-4098-8a0d-e2c106913d73 None admin] PATH_INFO: `/v3/users/b51a13f6fd5f429b80273d468335a1d8` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:21.977236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a69a96-4a51-4098-8a0d-e2c106913d73 None admin] RBAC: Authorizing `identity:get_user(user_id=b51a13f6fd5f429b80273d468335a1d8)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:21.980095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a69a96-4a51-4098-8a0d-e2c106913d73 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:21.988077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 122/247] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:21 2022] GET /identity/v3/users/b51a13f6fd5f429b80273d468335a1d8 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:21.996713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a69a96-4a51-4098-8a0d-e2c106913d73 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:22.010715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9382c62-d4e9-4aaa-81a8-eb1d8111b89a None admin] RBAC: Authorizing `identity:get_user(user_id=a3f295c9d0b6422b9f50eac9eb3c9f15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:22.012982 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9382c62-d4e9-4aaa-81a8-eb1d8111b89a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:22.015595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 126/248] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:21 2022] GET /identity/v3/users/a3f295c9d0b6422b9f50eac9eb3c9f15 => generated 297 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:22.017877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22ba0-3fad-41dc-a7d6-f6f0bd76215b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:22.018529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22ba0-3fad-41dc-a7d6-f6f0bd76215b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:22.019089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22ba0-3fad-41dc-a7d6-f6f0bd76215b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:22.020739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 127/249] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:22.022166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af22ba0-3fad-41dc-a7d6-f6f0bd76215b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:22.067572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:22.133346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a69a96-4a51-4098-8a0d-e2c106913d73 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:22.159225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af22ba0-3fad-41dc-a7d6-f6f0bd76215b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:22.185334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4c158-c015-4211-9e23-d060f9b908b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:22.185994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c158-c015-4211-9e23-d060f9b908b4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:22.186277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c158-c015-4211-9e23-d060f9b908b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:22.186554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c158-c015-4211-9e23-d060f9b908b4 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:22.187742 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4c158-c015-4211-9e23-d060f9b908b4 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:22.189366 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4c158-c015-4211-9e23-d060f9b908b4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:22.191589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 123/250] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:21 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:22.193076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ed14-2bb3-4c5e-8870-f629d3be3665 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:22.198877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ed14-2bb3-4c5e-8870-f629d3be3665 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:22.198877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ed14-2bb3-4c5e-8870-f629d3be3665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:22.214025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255b18d-b375-48aa-8e7a-6a05c63c9e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:22.214917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ad89ed14-2bb3-4c5e-8870-f629d3be3665 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:22.215160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255b18d-b375-48aa-8e7a-6a05c63c9e09 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:22.215612 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255b18d-b375-48aa-8e7a-6a05c63c9e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:22.216146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255b18d-b375-48aa-8e7a-6a05c63c9e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:22.216261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ad89ed14-2bb3-4c5e-8870-f629d3be3665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:22.277361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:22.340940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255b18d-b375-48aa-8e7a-6a05c63c9e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:22.341224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 124/251] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:22.341948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89ed14-2bb3-4c5e-8870-f629d3be3665 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:22.344028 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255b18d-b375-48aa-8e7a-6a05c63c9e09 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:22.433678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:22.454883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 128/252] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:22.455814 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255b18d-b375-48aa-8e7a-6a05c63c9e09 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:22.509174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:22.513445 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89ed14-2bb3-4c5e-8870-f629d3be3665 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:22.561906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa497e2c-55a3-4d7a-8fcd-ea4581a619ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:22.562847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa497e2c-55a3-4d7a-8fcd-ea4581a619ef None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:22.563127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa497e2c-55a3-4d7a-8fcd-ea4581a619ef None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:22.563400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa497e2c-55a3-4d7a-8fcd-ea4581a619ef None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389/users/b51a13f6fd5f429b80273d468335a1d8/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:22.566334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa497e2c-55a3-4d7a-8fcd-ea4581a619ef None admin] RBAC: Authorizing `identity:create_grant(project_id=6a913e1d666748509269614e0c708389, user_id=b51a13f6fd5f429b80273d468335a1d8, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:22.569239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255b18d-b375-48aa-8e7a-6a05c63c9e09 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:22.570161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa497e2c-55a3-4d7a-8fcd-ea4581a619ef None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:22.607744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983cb86-b491-4711-8f11-9d9eb2b8012f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:22.608363 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983cb86-b491-4711-8f11-9d9eb2b8012f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:22.608649 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983cb86-b491-4711-8f11-9d9eb2b8012f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:22.608931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983cb86-b491-4711-8f11-9d9eb2b8012f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:22.620579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983cb86-b491-4711-8f11-9d9eb2b8012f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1983cb86-b491-4711-8f11-9d9eb2b8012f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:22.626136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:22.626971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:22.627791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 129/253] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:23:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:22.629026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983cb86-b491-4711-8f11-9d9eb2b8012f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:22.642368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 125/254] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:23:22 2022] PUT /identity/v3/projects/6a913e1d666748509269614e0c708389/users/b51a13f6fd5f429b80273d468335a1d8/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:22.643771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f8fab-0878-4e21-a0e7-47535007a533 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:22.644056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f8fab-0878-4e21-a0e7-47535007a533 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:22.644338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f8fab-0878-4e21-a0e7-47535007a533 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:22.645867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 126/255] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:22.647559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f8fab-0878-4e21-a0e7-47535007a533 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:22.764272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f8fab-0878-4e21-a0e7-47535007a533 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:22.788452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:22.802551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983cb86-b491-4711-8f11-9d9eb2b8012f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:22.820611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:22.857158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8596df-f98f-41bd-ab82-f1201786d6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:22.857872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8596df-f98f-41bd-ab82-f1201786d6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:22.858095 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8596df-f98f-41bd-ab82-f1201786d6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:22.858371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8596df-f98f-41bd-ab82-f1201786d6d0 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:22.860872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8596df-f98f-41bd-ab82-f1201786d6d0 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:22.862405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8596df-f98f-41bd-ab82-f1201786d6d0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:22.864268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 130/256] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:22 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:22.867573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c1653-b01a-48e8-98ad-8e533cf52612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:22.868222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c1653-b01a-48e8-98ad-8e533cf52612 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:22.868555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c1653-b01a-48e8-98ad-8e533cf52612 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:22.868835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c1653-b01a-48e8-98ad-8e533cf52612 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:22.869610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c1653-b01a-48e8-98ad-8e533cf52612 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:22.869920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c1653-b01a-48e8-98ad-8e533cf52612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:22.871417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c1653-b01a-48e8-98ad-8e533cf52612 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:22.873319 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8926b28-ced9-4449-994e-97253ad95f53 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:22.873615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8926b28-ced9-4449-994e-97253ad95f53 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:22.873945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8926b28-ced9-4449-994e-97253ad95f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:22.885475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 127/257] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:22.886382 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c1653-b01a-48e8-98ad-8e533cf52612 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:22.902892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b8926b28-ced9-4449-994e-97253ad95f53 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:22.904112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b8926b28-ced9-4449-994e-97253ad95f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:23.015470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c1653-b01a-48e8-98ad-8e533cf52612 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:23.049818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c1077-75b2-4af8-a321-3b3869c0b020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:23.050446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c1077-75b2-4af8-a321-3b3869c0b020 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:23.050989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c1077-75b2-4af8-a321-3b3869c0b020 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:23.051096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 131/258] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:23.051535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c1077-75b2-4af8-a321-3b3869c0b020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:23.051695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8926b28-ced9-4449-994e-97253ad95f53 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:23.051786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c1077-75b2-4af8-a321-3b3869c0b020 None admin] RBAC: Adding query filter params (role.id=e2331f0635214139bdac452081a3f310, scope.project.id=6a913e1d666748509269614e0c708389, user.id=a3f295c9d0b6422b9f50eac9eb3c9f15) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:23.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c1077-75b2-4af8-a321-3b3869c0b020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:23.054448 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c1077-75b2-4af8-a321-3b3869c0b020 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:23.060306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 128/259] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:22 2022] GET /identity/v3/role_assignments?user.id=a3f295c9d0b6422b9f50eac9eb3c9f15&scope.project.id=6a913e1d666748509269614e0c708389&role.id=e2331f0635214139bdac452081a3f310 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:23.110674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c1077-75b2-4af8-a321-3b3869c0b020 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:23.172341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:23.280442 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c1077-75b2-4af8-a321-3b3869c0b020 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:23.306111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8926b28-ced9-4449-994e-97253ad95f53 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:23.324128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:23.348844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267c3f2-0f0b-49b7-a771-9cf8ce118c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:23.349533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267c3f2-0f0b-49b7-a771-9cf8ce118c0f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:23.349820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267c3f2-0f0b-49b7-a771-9cf8ce118c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:23.350110 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267c3f2-0f0b-49b7-a771-9cf8ce118c0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:23.367560 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267c3f2-0f0b-49b7-a771-9cf8ce118c0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8267c3f2-0f0b-49b7-a771-9cf8ce118c0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:23.373826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:23.374701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:23.375584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:23.376673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdd12c-e09f-44b3-9937-b4bc466004c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:23.380005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdd12c-e09f-44b3-9937-b4bc466004c1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:23.380005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdd12c-e09f-44b3-9937-b4bc466004c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:23.380005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdd12c-e09f-44b3-9937-b4bc466004c1 None admin] PATH_INFO: `/v3/users/8f37dff953764cfdb5bea1432e35ba87` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:23.380005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabdd12c-e09f-44b3-9937-b4bc466004c1 None admin] RBAC: Authorizing `identity:get_user(user_id=8f37dff953764cfdb5bea1432e35ba87)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:23.380430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabdd12c-e09f-44b3-9937-b4bc466004c1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:23.382828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 132/261] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:23 2022] GET /identity/v3/users/8f37dff953764cfdb5bea1432e35ba87 => generated 283 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:23.383654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdd12c-e09f-44b3-9937-b4bc466004c1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:23.384069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 129/261] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:23.388278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267c3f2-0f0b-49b7-a771-9cf8ce118c0f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:23.564523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdd12c-e09f-44b3-9937-b4bc466004c1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:23.587776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:23.603100 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267c3f2-0f0b-49b7-a771-9cf8ce118c0f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:23.656794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea905-577b-4853-8e38-29b22722ed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:23.657350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea905-577b-4853-8e38-29b22722ed31 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:23.657635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea905-577b-4853-8e38-29b22722ed31 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:23.657910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea905-577b-4853-8e38-29b22722ed31 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:23.658637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ea905-577b-4853-8e38-29b22722ed31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:23.658944 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ea905-577b-4853-8e38-29b22722ed31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:23.660343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ea905-577b-4853-8e38-29b22722ed31 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:23.675632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 133/262] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:23.681659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea905-577b-4853-8e38-29b22722ed31 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:23.688686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1430c-01d6-40c3-b9cb-c27abfb5dfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:23.689328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1430c-01d6-40c3-b9cb-c27abfb5dfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:23.689614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1430c-01d6-40c3-b9cb-c27abfb5dfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:23.689892 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1430c-01d6-40c3-b9cb-c27abfb5dfd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:23.788345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:23.834685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1430c-01d6-40c3-b9cb-c27abfb5dfd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:23.836203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1430c-01d6-40c3-b9cb-c27abfb5dfd7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:23.931965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea905-577b-4853-8e38-29b22722ed31 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:24.021353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb307a-8b60-4cff-91cf-03e9750f34d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:24.022019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb307a-8b60-4cff-91cf-03e9750f34d9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:24.022307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb307a-8b60-4cff-91cf-03e9750f34d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:24.022584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb307a-8b60-4cff-91cf-03e9750f34d9 None admin] PATH_INFO: `/v3/users/8f37dff953764cfdb5bea1432e35ba87` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:24.027341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb307a-8b60-4cff-91cf-03e9750f34d9 None admin] RBAC: Authorizing `identity:get_user(user_id=8f37dff953764cfdb5bea1432e35ba87)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:24.028845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb307a-8b60-4cff-91cf-03e9750f34d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:24.030652 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 134/263] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:23 2022] GET /identity/v3/users/8f37dff953764cfdb5bea1432e35ba87 => generated 283 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:24.036256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb307a-8b60-4cff-91cf-03e9750f34d9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:24.047443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:24.064211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 130/264] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 07:23:24.070578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1430c-01d6-40c3-b9cb-c27abfb5dfd7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:24.249092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1430c-01d6-40c3-b9cb-c27abfb5dfd7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:24.266133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb307a-8b60-4cff-91cf-03e9750f34d9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:24.278284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:24.306348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e5031-fc23-465e-bd28-7630ee12b531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:24.307006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5031-fc23-465e-bd28-7630ee12b531 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:24.307349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5031-fc23-465e-bd28-7630ee12b531 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:24.307634 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5031-fc23-465e-bd28-7630ee12b531 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:24.310845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e5031-fc23-465e-bd28-7630ee12b531 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:24.312373 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e5031-fc23-465e-bd28-7630ee12b531 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:24.314378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 131/265] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:24 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:24.324006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e5031-fc23-465e-bd28-7630ee12b531 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:24.332699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef4187-aa25-4f98-a12a-104bde590c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:24.333733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef4187-aa25-4f98-a12a-104bde590c89 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:24.334712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef4187-aa25-4f98-a12a-104bde590c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:24.335208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef4187-aa25-4f98-a12a-104bde590c89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:24.441371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef4187-aa25-4f98-a12a-104bde590c89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:24.443786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef4187-aa25-4f98-a12a-104bde590c89 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:24.444341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:24.490454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e5031-fc23-465e-bd28-7630ee12b531 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:24.576109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381069f5-a55d-47d8-94e6-e08946347b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:24.577088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381069f5-a55d-47d8-94e6-e08946347b16 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:24.577578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381069f5-a55d-47d8-94e6-e08946347b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:24.578083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381069f5-a55d-47d8-94e6-e08946347b16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:24.579127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381069f5-a55d-47d8-94e6-e08946347b16 None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=6a913e1d666748509269614e0c708389, user.id=8f37dff953764cfdb5bea1432e35ba87) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:24.579646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381069f5-a55d-47d8-94e6-e08946347b16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:24.581661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381069f5-a55d-47d8-94e6-e08946347b16 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:24.597378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 132/266] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:24 2022] GET /identity/v3/role_assignments?user.id=8f37dff953764cfdb5bea1432e35ba87&scope.project.id=6a913e1d666748509269614e0c708389&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:24.612267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 135/267] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 07:23:24.618619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381069f5-a55d-47d8-94e6-e08946347b16 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:24.647957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d294d91-2c20-474c-9b51-ea266950557f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:24.648279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d294d91-2c20-474c-9b51-ea266950557f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:24.648556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d294d91-2c20-474c-9b51-ea266950557f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:24.650172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 136/268] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:24.655361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c389d3-1a99-4d2f-bef9-1926dca3be46 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:24.655637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c389d3-1a99-4d2f-bef9-1926dca3be46 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:24.655910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c389d3-1a99-4d2f-bef9-1926dca3be46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:24.675077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:24.685547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-21c389d3-1a99-4d2f-bef9-1926dca3be46 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:24.690044 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21c389d3-1a99-4d2f-bef9-1926dca3be46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:24.805533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381069f5-a55d-47d8-94e6-e08946347b16 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:24.850245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:24.861377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dcff2-4297-448d-aa74-67f9b830d07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:24.862155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dcff2-4297-448d-aa74-67f9b830d07c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:24.862445 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dcff2-4297-448d-aa74-67f9b830d07c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:24.862760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dcff2-4297-448d-aa74-67f9b830d07c None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:24.862922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 137/269] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:24.863955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dcff2-4297-448d-aa74-67f9b830d07c None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:24.866005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dcff2-4297-448d-aa74-67f9b830d07c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:24.879475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 133/270] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:24 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:24.879600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c389d3-1a99-4d2f-bef9-1926dca3be46 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:24.894620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dcff2-4297-448d-aa74-67f9b830d07c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:25.028437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dcff2-4297-448d-aa74-67f9b830d07c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:25.064952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9c680-fcb6-4feb-aaf2-5b918fbee1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:25.065588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9c680-fcb6-4feb-aaf2-5b918fbee1ce None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.065868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9c680-fcb6-4feb-aaf2-5b918fbee1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.066143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9c680-fcb6-4feb-aaf2-5b918fbee1ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.081290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de9c680-fcb6-4feb-aaf2-5b918fbee1ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3de9c680-fcb6-4feb-aaf2-5b918fbee1ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:25.092999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:25.093930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:25.094736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 134/271] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:25.102839 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9c680-fcb6-4feb-aaf2-5b918fbee1ce None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:25.107188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c389d3-1a99-4d2f-bef9-1926dca3be46 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:25.146961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af781398-540e-4dcc-ae79-921590bda93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:25.147576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af781398-540e-4dcc-ae79-921590bda93d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.147854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af781398-540e-4dcc-ae79-921590bda93d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.148154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af781398-540e-4dcc-ae79-921590bda93d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.148994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af781398-540e-4dcc-ae79-921590bda93d None admin] RBAC: Adding query filter params (role.id=e3c5a858a45c41db8ff3e883788d75bc, scope.project.id=6a913e1d666748509269614e0c708389, user.id=8f37dff953764cfdb5bea1432e35ba87) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:25.149308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af781398-540e-4dcc-ae79-921590bda93d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:25.150748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af781398-540e-4dcc-ae79-921590bda93d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:25.153678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:25.157935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 138/272] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:24 2022] GET /identity/v3/role_assignments?user.id=8f37dff953764cfdb5bea1432e35ba87&scope.project.id=6a913e1d666748509269614e0c708389&role.id=e3c5a858a45c41db8ff3e883788d75bc => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:25.159178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0826f5-9730-455b-8f88-b379c71bf245 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.159463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0826f5-9730-455b-8f88-b379c71bf245 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.159736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0826f5-9730-455b-8f88-b379c71bf245 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.161212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 139/273] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:25.166785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720d4a0-eecb-4b4f-a23a-c9b828da85c6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.167361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720d4a0-eecb-4b4f-a23a-c9b828da85c6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.167817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720d4a0-eecb-4b4f-a23a-c9b828da85c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.174109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:25.208264 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5720d4a0-eecb-4b4f-a23a-c9b828da85c6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:25.210037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5720d4a0-eecb-4b4f-a23a-c9b828da85c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:25.273873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9c680-fcb6-4feb-aaf2-5b918fbee1ce None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:25.350868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382271eb-46d6-4276-b375-b7133a321354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:25.351502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382271eb-46d6-4276-b375-b7133a321354 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.351784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382271eb-46d6-4276-b375-b7133a321354 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.352093 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382271eb-46d6-4276-b375-b7133a321354 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.352827 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382271eb-46d6-4276-b375-b7133a321354 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:25.353182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382271eb-46d6-4276-b375-b7133a321354 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:25.354517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382271eb-46d6-4276-b375-b7133a321354 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:25.363840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 140/274] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:25.365779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a635f-e57c-4b48-902e-2a66e7f22531 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.366130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a635f-e57c-4b48-902e-2a66e7f22531 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.366221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 135/275] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:25.366304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a635f-e57c-4b48-902e-2a66e7f22531 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.367678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 141/276] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:25.373310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ce5ac-3174-4e76-9bea-b0348fad5d71 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.373601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ce5ac-3174-4e76-9bea-b0348fad5d71 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.373878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ce5ac-3174-4e76-9bea-b0348fad5d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.378697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a635f-e57c-4b48-902e-2a66e7f22531 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:25.395035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fa1ce5ac-3174-4e76-9bea-b0348fad5d71 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:25.396444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fa1ce5ac-3174-4e76-9bea-b0348fad5d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:25.472568 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 136/277] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:25.473954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ce5ac-3174-4e76-9bea-b0348fad5d71 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:25.512582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a635f-e57c-4b48-902e-2a66e7f22531 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:25.528042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:25.544180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:25.550642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6f102-a59c-49ac-b6a9-cabe7a0b7729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:25.551361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6f102-a59c-49ac-b6a9-cabe7a0b7729 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.551661 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6f102-a59c-49ac-b6a9-cabe7a0b7729 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.551893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6f102-a59c-49ac-b6a9-cabe7a0b7729 None admin] PATH_INFO: `/v3/users/081e1b7ff09444cdaed2b2d68248b779` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.566067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6f102-a59c-49ac-b6a9-cabe7a0b7729 None admin] RBAC: Authorizing `identity:get_user(user_id=081e1b7ff09444cdaed2b2d68248b779)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:25.567620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6f102-a59c-49ac-b6a9-cabe7a0b7729 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:25.570519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 142/278] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:25 2022] GET /identity/v3/users/081e1b7ff09444cdaed2b2d68248b779 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:25.571088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6f102-a59c-49ac-b6a9-cabe7a0b7729 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:25.593102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ce5ac-3174-4e76-9bea-b0348fad5d71 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:25.629491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c8015-888a-43ab-870f-a9f160e00cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:25.630053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8015-888a-43ab-870f-a9f160e00cdf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.630910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8015-888a-43ab-870f-a9f160e00cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.631278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8015-888a-43ab-870f-a9f160e00cdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.632915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c8015-888a-43ab-870f-a9f160e00cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:25.634344 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c8015-888a-43ab-870f-a9f160e00cdf None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:25.636199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 137/279] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:25.636944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c8015-888a-43ab-870f-a9f160e00cdf None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:25.682071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6f102-a59c-49ac-b6a9-cabe7a0b7729 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:25.718783 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6aba6-7974-4b06-a0ac-152a288eb097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:25.719423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6aba6-7974-4b06-a0ac-152a288eb097 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.719706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6aba6-7974-4b06-a0ac-152a288eb097 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.720010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6aba6-7974-4b06-a0ac-152a288eb097 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.730486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6aba6-7974-4b06-a0ac-152a288eb097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f5a6aba6-7974-4b06-a0ac-152a288eb097 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:25.735644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:25.736570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:25.737375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 143/280] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:25.738603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6aba6-7974-4b06-a0ac-152a288eb097 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:25.758254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c8015-888a-43ab-870f-a9f160e00cdf None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:25.776146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:25.792983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:25.796174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8027efe-992c-48ea-be40-79631898eb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:25.796792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8027efe-992c-48ea-be40-79631898eb6d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.797346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8027efe-992c-48ea-be40-79631898eb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.797346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8027efe-992c-48ea-be40-79631898eb6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.798114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8027efe-992c-48ea-be40-79631898eb6d None admin] RBAC: Adding query filter params (role.id=525e499f44594268b4f362c62bd35e00, user.id=081e1b7ff09444cdaed2b2d68248b779, scope.system=all) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:25.798423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8027efe-992c-48ea-be40-79631898eb6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:25.799851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8027efe-992c-48ea-be40-79631898eb6d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:25.805050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 138/281] 10.222.0.64 () {56 vars in 1308 bytes} [Thu Aug 4 07:23:25 2022] GET /identity/v3/role_assignments?user.id=081e1b7ff09444cdaed2b2d68248b779&scope.system=all&role.id=525e499f44594268b4f362c62bd35e00 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:25.817108 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8027efe-992c-48ea-be40-79631898eb6d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:25.852963 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6aba6-7974-4b06-a0ac-152a288eb097 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:25.893756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd85f9-611c-4b92-9b38-b0c78d503a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:25.894309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd85f9-611c-4b92-9b38-b0c78d503a4b None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.894639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd85f9-611c-4b92-9b38-b0c78d503a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.894917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd85f9-611c-4b92-9b38-b0c78d503a4b None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.895624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd85f9-611c-4b92-9b38-b0c78d503a4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:25.898123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd85f9-611c-4b92-9b38-b0c78d503a4b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:25.960523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8027efe-992c-48ea-be40-79631898eb6d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:25.996145 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d1c87-b5cf-4812-b95d-4abe5324e0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:25.996770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d1c87-b5cf-4812-b95d-4abe5324e0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.997050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d1c87-b5cf-4812-b95d-4abe5324e0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:25.997323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d1c87-b5cf-4812-b95d-4abe5324e0a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:25.998066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d1c87-b5cf-4812-b95d-4abe5324e0a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:25.998162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 144/282] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:23:25 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:23:25.998372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d1c87-b5cf-4812-b95d-4abe5324e0a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:25.999511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62894de4-a3ed-4b06-9682-0300850aabd5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:25.999746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d1c87-b5cf-4812-b95d-4abe5324e0a9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:25.999840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62894de4-a3ed-4b06-9682-0300850aabd5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:26.000151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62894de4-a3ed-4b06-9682-0300850aabd5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:26.004025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 145/283] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:26.007425 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94983352-956f-4039-8edb-ec402bbe513a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:26.007711 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94983352-956f-4039-8edb-ec402bbe513a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:26.008012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94983352-956f-4039-8edb-ec402bbe513a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:26.010167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 139/284] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:26.015521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d1c87-b5cf-4812-b95d-4abe5324e0a9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:26.027011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-94983352-956f-4039-8edb-ec402bbe513a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:26.028295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-94983352-956f-4039-8edb-ec402bbe513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:26.078348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:26.108521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 146/285] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:26.131839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94983352-956f-4039-8edb-ec402bbe513a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:26.158698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d1c87-b5cf-4812-b95d-4abe5324e0a9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:26.171294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:26.208003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330a7cf-e8d9-4550-91af-e49fdc24d877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:26.210122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330a7cf-e8d9-4550-91af-e49fdc24d877 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:26.210668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330a7cf-e8d9-4550-91af-e49fdc24d877 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:26.211403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330a7cf-e8d9-4550-91af-e49fdc24d877 None admin] PATH_INFO: `/v3/users/b51a13f6fd5f429b80273d468335a1d8` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:26.212978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330a7cf-e8d9-4550-91af-e49fdc24d877 None admin] RBAC: Authorizing `identity:get_user(user_id=b51a13f6fd5f429b80273d468335a1d8)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:26.215402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330a7cf-e8d9-4550-91af-e49fdc24d877 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:26.217069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 140/286] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:26 2022] GET /identity/v3/users/b51a13f6fd5f429b80273d468335a1d8 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:26.225214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330a7cf-e8d9-4550-91af-e49fdc24d877 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:26.248595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94983352-956f-4039-8edb-ec402bbe513a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:26.296284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd0acd-80f6-455d-abc7-d0eb20ee3531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:26.296969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd0acd-80f6-455d-abc7-d0eb20ee3531 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:26.297257 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd0acd-80f6-455d-abc7-d0eb20ee3531 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:26.297634 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd0acd-80f6-455d-abc7-d0eb20ee3531 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:26.310260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd0acd-80f6-455d-abc7-d0eb20ee3531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-dfdd0acd-80f6-455d-abc7-d0eb20ee3531 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:26.315759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:26.316664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:26.317461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 147/287] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:26.325249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd0acd-80f6-455d-abc7-d0eb20ee3531 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:26.348117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330a7cf-e8d9-4550-91af-e49fdc24d877 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:26.359771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:26.392953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183597b9-ea4a-4977-a8c8-3996f4ff2bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:26.394261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183597b9-ea4a-4977-a8c8-3996f4ff2bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:26.394695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183597b9-ea4a-4977-a8c8-3996f4ff2bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:26.395376 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183597b9-ea4a-4977-a8c8-3996f4ff2bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:26.449027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:26.459153 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd0acd-80f6-455d-abc7-d0eb20ee3531 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:26.478520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183597b9-ea4a-4977-a8c8-3996f4ff2bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:26.478738 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183597b9-ea4a-4977-a8c8-3996f4ff2bd2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:26.506991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8114156-3e23-4c49-9c35-a625bcd4c9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:26.506991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8114156-3e23-4c49-9c35-a625bcd4c9dc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:26.507312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8114156-3e23-4c49-9c35-a625bcd4c9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:26.507565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8114156-3e23-4c49-9c35-a625bcd4c9dc None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:26.509212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8114156-3e23-4c49-9c35-a625bcd4c9dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:26.509524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8114156-3e23-4c49-9c35-a625bcd4c9dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:26.510938 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8114156-3e23-4c49-9c35-a625bcd4c9dc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:26.521662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 148/288] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:26.526916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8114156-3e23-4c49-9c35-a625bcd4c9dc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:26.605070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 141/289] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 07:23:26.613781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183597b9-ea4a-4977-a8c8-3996f4ff2bd2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:26.631068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:26.690331 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8114156-3e23-4c49-9c35-a625bcd4c9dc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:26.739334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf016f2-fa6a-4d2c-8322-e14f806ba348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:26.742621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf016f2-fa6a-4d2c-8322-e14f806ba348 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:26.742915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf016f2-fa6a-4d2c-8322-e14f806ba348 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:26.743873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf016f2-fa6a-4d2c-8322-e14f806ba348 None admin] PATH_INFO: `/v3/users/a3f295c9d0b6422b9f50eac9eb3c9f15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:26.748934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf016f2-fa6a-4d2c-8322-e14f806ba348 None admin] RBAC: Authorizing `identity:get_user(user_id=a3f295c9d0b6422b9f50eac9eb3c9f15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:26.750387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf016f2-fa6a-4d2c-8322-e14f806ba348 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:26.752009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 149/290] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:26 2022] GET /identity/v3/users/a3f295c9d0b6422b9f50eac9eb3c9f15 => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:26.761538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf016f2-fa6a-4d2c-8322-e14f806ba348 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:26.812354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:26.819133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183597b9-ea4a-4977-a8c8-3996f4ff2bd2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:26.854015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551db36-58a0-4cf0-a343-d8d8653e8786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:26.854670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551db36-58a0-4cf0-a343-d8d8653e8786 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:26.855058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551db36-58a0-4cf0-a343-d8d8653e8786 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:26.855343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551db36-58a0-4cf0-a343-d8d8653e8786 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:26.856539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551db36-58a0-4cf0-a343-d8d8653e8786 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:26.857979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551db36-58a0-4cf0-a343-d8d8653e8786 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:26.860330 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 142/291] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:26 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:26.866911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551db36-58a0-4cf0-a343-d8d8653e8786 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:26.952226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:26.956070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf016f2-fa6a-4d2c-8322-e14f806ba348 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:26.996270 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551db36-58a0-4cf0-a343-d8d8653e8786 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:27.034575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688ce36-3760-4e76-a062-8ba2265a1ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:27.035195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688ce36-3760-4e76-a062-8ba2265a1ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:27.035489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688ce36-3760-4e76-a062-8ba2265a1ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:27.035774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688ce36-3760-4e76-a062-8ba2265a1ea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:27.036682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0688ce36-3760-4e76-a062-8ba2265a1ea2 None admin] RBAC: Adding query filter params (role.id=525e499f44594268b4f362c62bd35e00, scope.project.id=6a913e1d666748509269614e0c708389, user.id=b51a13f6fd5f429b80273d468335a1d8) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:27.036999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0688ce36-3760-4e76-a062-8ba2265a1ea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:27.037455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0087366-bb90-4fd5-85fc-e36f041c61b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:27.038109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087366-bb90-4fd5-85fc-e36f041c61b2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:27.038394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087366-bb90-4fd5-85fc-e36f041c61b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:27.038595 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0688ce36-3760-4e76-a062-8ba2265a1ea2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:27.038691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087366-bb90-4fd5-85fc-e36f041c61b2 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:27.044745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0087366-bb90-4fd5-85fc-e36f041c61b2 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:27.045156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 143/292] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:26 2022] GET /identity/v3/role_assignments?user.id=b51a13f6fd5f429b80273d468335a1d8&scope.project.id=6a913e1d666748509269614e0c708389&role.id=525e499f44594268b4f362c62bd35e00 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:27.046216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0087366-bb90-4fd5-85fc-e36f041c61b2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:27.047907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 150/293] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:26 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:27.054805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688ce36-3760-4e76-a062-8ba2265a1ea2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:27.170727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688ce36-3760-4e76-a062-8ba2265a1ea2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:27.209422 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3d8fd-ecfa-497b-a5d3-86b7bff8fa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:27.210125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3d8fd-ecfa-497b-a5d3-86b7bff8fa77 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:27.210417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3d8fd-ecfa-497b-a5d3-86b7bff8fa77 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:27.210700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3d8fd-ecfa-497b-a5d3-86b7bff8fa77 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389/users/a3f295c9d0b6422b9f50eac9eb3c9f15/roles/e2331f0635214139bdac452081a3f310` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:27.212785 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3d8fd-ecfa-497b-a5d3-86b7bff8fa77 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a913e1d666748509269614e0c708389, user_id=a3f295c9d0b6422b9f50eac9eb3c9f15, role_id=e2331f0635214139bdac452081a3f310)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:27.214266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3d8fd-ecfa-497b-a5d3-86b7bff8fa77 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:27.331484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:27.336088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 144/294] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:23:27 2022] PUT /identity/v3/projects/6a913e1d666748509269614e0c708389/users/a3f295c9d0b6422b9f50eac9eb3c9f15/roles/e2331f0635214139bdac452081a3f310 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:28.019149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213a6aa-f049-4517-ac2c-f5a1eff154d9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:28.019703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213a6aa-f049-4517-ac2c-f5a1eff154d9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:28.019703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213a6aa-f049-4517-ac2c-f5a1eff154d9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:28.021408 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 151/295] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:28.026615 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285af79-7f30-41fd-8cd3-0748c698c05f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:28.026904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285af79-7f30-41fd-8cd3-0748c698c05f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:28.027188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285af79-7f30-41fd-8cd3-0748c698c05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:28.047208 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0285af79-7f30-41fd-8cd3-0748c698c05f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:28.048467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0285af79-7f30-41fd-8cd3-0748c698c05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:28.120298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 145/296] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:28.144066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213a6aa-f049-4517-ac2c-f5a1eff154d9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:28.293824 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:28.303974 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213a6aa-f049-4517-ac2c-f5a1eff154d9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:28.339049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67110ac3-fa19-4910-bd4f-2af597e2aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:28.339788 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67110ac3-fa19-4910-bd4f-2af597e2aef3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:28.340316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67110ac3-fa19-4910-bd4f-2af597e2aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:28.340848 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67110ac3-fa19-4910-bd4f-2af597e2aef3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:28.352447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67110ac3-fa19-4910-bd4f-2af597e2aef3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-67110ac3-fa19-4910-bd4f-2af597e2aef3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:28.357447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:28.358309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:28.359110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 152/297] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:28.367139 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285af79-7f30-41fd-8cd3-0748c698c05f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:28.442442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37869058-0475-4246-b237-717f9301dd4c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:28.443433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37869058-0475-4246-b237-717f9301dd4c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:28.443948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37869058-0475-4246-b237-717f9301dd4c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:28.445935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 153/298] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:28.451061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f6057-58d7-411b-b022-e0ba341bd75c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:28.451582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f6057-58d7-411b-b022-e0ba341bd75c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:28.451958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f6057-58d7-411b-b022-e0ba341bd75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:28.471861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cb2f6057-58d7-411b-b022-e0ba341bd75c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:28.473417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cb2f6057-58d7-411b-b022-e0ba341bd75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:28.473661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285af79-7f30-41fd-8cd3-0748c698c05f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:28.520368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:28.524598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9fa92-7ab3-4bd9-85be-9e927cc53c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:28.525243 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9fa92-7ab3-4bd9-85be-9e927cc53c0a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:28.525531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9fa92-7ab3-4bd9-85be-9e927cc53c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:28.525808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9fa92-7ab3-4bd9-85be-9e927cc53c0a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:28.526602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9fa92-7ab3-4bd9-85be-9e927cc53c0a None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:28.526913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9fa92-7ab3-4bd9-85be-9e927cc53c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:28.528363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9fa92-7ab3-4bd9-85be-9e927cc53c0a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:28.540569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 146/299] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:28.546404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9fa92-7ab3-4bd9-85be-9e927cc53c0a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:28.575234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 154/300] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:28.593909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f6057-58d7-411b-b022-e0ba341bd75c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:28.652291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9fa92-7ab3-4bd9-85be-9e927cc53c0a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:28.683790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:28.685954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab9fcf-18ca-4fd2-9761-b4d04ebe117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:28.686600 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab9fcf-18ca-4fd2-9761-b4d04ebe117d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:28.686884 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab9fcf-18ca-4fd2-9761-b4d04ebe117d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:28.687174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab9fcf-18ca-4fd2-9761-b4d04ebe117d None admin] PATH_INFO: `/v3/users/081e1b7ff09444cdaed2b2d68248b779` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:28.688375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab9fcf-18ca-4fd2-9761-b4d04ebe117d None admin] RBAC: Authorizing `identity:get_user(user_id=081e1b7ff09444cdaed2b2d68248b779)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:28.689801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab9fcf-18ca-4fd2-9761-b4d04ebe117d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:28.691488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 147/301] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:28 2022] GET /identity/v3/users/081e1b7ff09444cdaed2b2d68248b779 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:28.696752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb9ce3-01a9-4041-8003-6199a3c34c54 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:28.696752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb9ce3-01a9-4041-8003-6199a3c34c54 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:28.696752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb9ce3-01a9-4041-8003-6199a3c34c54 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:28.696752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 148/302] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:28.698130 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb9ce3-01a9-4041-8003-6199a3c34c54 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:28.712485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f6057-58d7-411b-b022-e0ba341bd75c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:28.748069 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c9dc4-a26d-43f7-a06c-0c18fcad9358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:28.748708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c9dc4-a26d-43f7-a06c-0c18fcad9358 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:28.748988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c9dc4-a26d-43f7-a06c-0c18fcad9358 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:28.749258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c9dc4-a26d-43f7-a06c-0c18fcad9358 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:28.760557 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c9dc4-a26d-43f7-a06c-0c18fcad9358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:28.761042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9c9c9dc4-a26d-43f7-a06c-0c18fcad9358 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:28.766644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:28.774052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 155/303] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:28.774842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52407-210f-4d9c-bec4-66c9206e730a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:28.775617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52407-210f-4d9c-bec4-66c9206e730a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:28.775617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52407-210f-4d9c-bec4-66c9206e730a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:28.790569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-aab52407-210f-4d9c-bec4-66c9206e730a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:28.791745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aab52407-210f-4d9c-bec4-66c9206e730a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:28.809573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb9ce3-01a9-4041-8003-6199a3c34c54 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:28.850316 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465639c7-25ef-46a4-a2f6-185dccc1d0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:28.850968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465639c7-25ef-46a4-a2f6-185dccc1d0c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:28.851250 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465639c7-25ef-46a4-a2f6-185dccc1d0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:28.851685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465639c7-25ef-46a4-a2f6-185dccc1d0c6 None admin] PATH_INFO: `/v3/system/users/081e1b7ff09444cdaed2b2d68248b779/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:28.853101 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465639c7-25ef-46a4-a2f6-185dccc1d0c6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=081e1b7ff09444cdaed2b2d68248b779, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:28.854708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465639c7-25ef-46a4-a2f6-185dccc1d0c6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:28.859680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 156/304] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:28.860637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab52407-210f-4d9c-bec4-66c9206e730a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:28.932624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:28.941871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 149/305] 10.222.0.64 () {58 vars in 1334 bytes} [Thu Aug 4 07:23:28 2022] PUT /identity/v3/system/users/081e1b7ff09444cdaed2b2d68248b779/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:28.948221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465639c7-25ef-46a4-a2f6-185dccc1d0c6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:28.983110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab52407-210f-4d9c-bec4-66c9206e730a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:29.025868 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cad6b-7d81-466e-a774-d58b2b467b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:29.026572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cad6b-7d81-466e-a774-d58b2b467b88 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:29.026949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cad6b-7d81-466e-a774-d58b2b467b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:29.027143 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cad6b-7d81-466e-a774-d58b2b467b88 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:29.027911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9cad6b-7d81-466e-a774-d58b2b467b88 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:29.028295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9cad6b-7d81-466e-a774-d58b2b467b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:29.030123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9cad6b-7d81-466e-a774-d58b2b467b88 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:29.041372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 157/306] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:29.046593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cad6b-7d81-466e-a774-d58b2b467b88 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:29.082128 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465639c7-25ef-46a4-a2f6-185dccc1d0c6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:29.121245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:29.140194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82c70d-87af-4bb8-a890-9a72d2120de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:29.141746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82c70d-87af-4bb8-a890-9a72d2120de0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:29.142042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82c70d-87af-4bb8-a890-9a72d2120de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:29.142326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82c70d-87af-4bb8-a890-9a72d2120de0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:29.143521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82c70d-87af-4bb8-a890-9a72d2120de0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:29.145078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82c70d-87af-4bb8-a890-9a72d2120de0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:29.147946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 150/307] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:29.154618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82c70d-87af-4bb8-a890-9a72d2120de0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:29.187896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cad6b-7d81-466e-a774-d58b2b467b88 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:29.236810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa753c21-6864-4ffa-84f2-db4220849c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:29.237704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa753c21-6864-4ffa-84f2-db4220849c23 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:29.238162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa753c21-6864-4ffa-84f2-db4220849c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:29.238585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa753c21-6864-4ffa-84f2-db4220849c23 None admin] PATH_INFO: `/v3/users/5ec4a3dcb6804da8aa00ad2efa718450` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:29.242043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:29.259509 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa753c21-6864-4ffa-84f2-db4220849c23 None admin] RBAC: Authorizing `identity:get_user(user_id=5ec4a3dcb6804da8aa00ad2efa718450)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:29.261783 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa753c21-6864-4ffa-84f2-db4220849c23 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:29.264876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 158/308] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:29 2022] GET /identity/v3/users/5ec4a3dcb6804da8aa00ad2efa718450 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:29.272025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82c70d-87af-4bb8-a890-9a72d2120de0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:29.273132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa753c21-6864-4ffa-84f2-db4220849c23 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:29.312451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27ceec-c392-42b7-8309-922167f5e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:29.313234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27ceec-c392-42b7-8309-922167f5e7f8 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:29.313605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27ceec-c392-42b7-8309-922167f5e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:29.313996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27ceec-c392-42b7-8309-922167f5e7f8 None admin] PATH_INFO: `/v3/groups` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:29.314769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27ceec-c392-42b7-8309-922167f5e7f8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:29.316464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27ceec-c392-42b7-8309-922167f5e7f8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:29.391925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa753c21-6864-4ffa-84f2-db4220849c23 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:29.401091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 151/309] 10.222.0.64 () {60 vars in 1134 bytes} [Thu Aug 4 07:23:29 2022] POST /identity/v3/groups => generated 229 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:23:29.432401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093a20-26ba-4e1d-9c7b-cb6bbb024cf6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:29.433025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093a20-26ba-4e1d-9c7b-cb6bbb024cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:29.433510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093a20-26ba-4e1d-9c7b-cb6bbb024cf6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:29.435724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 152/310] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:23:29.441082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288c219-cb9c-414c-b3c9-e1ff3f35f62e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:29.441496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288c219-cb9c-414c-b3c9-e1ff3f35f62e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:29.441889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288c219-cb9c-414c-b3c9-e1ff3f35f62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:29.443612 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f980c9f-e4ca-421d-a9da-f1332bd5dcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:29.444458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f980c9f-e4ca-421d-a9da-f1332bd5dcb2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:29.444893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f980c9f-e4ca-421d-a9da-f1332bd5dcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:29.445342 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f980c9f-e4ca-421d-a9da-f1332bd5dcb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:29.446308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f980c9f-e4ca-421d-a9da-f1332bd5dcb2 None admin] RBAC: Adding query filter params (role.id=e2331f0635214139bdac452081a3f310, user.id=5ec4a3dcb6804da8aa00ad2efa718450, scope.system=all) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:29.446626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f980c9f-e4ca-421d-a9da-f1332bd5dcb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:29.448142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f980c9f-e4ca-421d-a9da-f1332bd5dcb2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:29.454443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 159/311] 10.222.0.64 () {56 vars in 1308 bytes} [Thu Aug 4 07:23:29 2022] GET /identity/v3/role_assignments?user.id=5ec4a3dcb6804da8aa00ad2efa718450&scope.system=all&role.id=e2331f0635214139bdac452081a3f310 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:29.466150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1288c219-cb9c-414c-b3c9-e1ff3f35f62e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:29.467647 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1288c219-cb9c-414c-b3c9-e1ff3f35f62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:29.536306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:29.552464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 153/312] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:29.577755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f980c9f-e4ca-421d-a9da-f1332bd5dcb2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:29.648430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:29.694233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f980c9f-e4ca-421d-a9da-f1332bd5dcb2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:29.730005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c51bfa-cc6c-4c30-a58d-3dcccf666d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:29.730648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c51bfa-cc6c-4c30-a58d-3dcccf666d36 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:29.730989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c51bfa-cc6c-4c30-a58d-3dcccf666d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:29.731268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c51bfa-cc6c-4c30-a58d-3dcccf666d36 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:29.742563 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c51bfa-cc6c-4c30-a58d-3dcccf666d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-69c51bfa-cc6c-4c30-a58d-3dcccf666d36 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:29.748059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:29.748917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:29.749683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 160/313] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:29.755092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288c219-cb9c-414c-b3c9-e1ff3f35f62e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:29.862559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288c219-cb9c-414c-b3c9-e1ff3f35f62e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:29.897439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261bfc7-2cda-40f9-b58e-32cec602fcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:29.898056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261bfc7-2cda-40f9-b58e-32cec602fcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:29.898335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261bfc7-2cda-40f9-b58e-32cec602fcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:29.898667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261bfc7-2cda-40f9-b58e-32cec602fcb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:29.899326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261bfc7-2cda-40f9-b58e-32cec602fcb4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:29.899629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261bfc7-2cda-40f9-b58e-32cec602fcb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:29.901014 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261bfc7-2cda-40f9-b58e-32cec602fcb4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:29.909798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 154/314] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:29.915000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c51bfa-cc6c-4c30-a58d-3dcccf666d36 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:30.027002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c51bfa-cc6c-4c30-a58d-3dcccf666d36 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:30.044934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:30.062984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4f6d9-954d-4ea5-89aa-c94adbbcbd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:30.063632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4f6d9-954d-4ea5-89aa-c94adbbcbd96 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:30.066676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4f6d9-954d-4ea5-89aa-c94adbbcbd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:30.066676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4f6d9-954d-4ea5-89aa-c94adbbcbd96 None admin] PATH_INFO: `/v3/users/a3f295c9d0b6422b9f50eac9eb3c9f15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:30.066676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4f6d9-954d-4ea5-89aa-c94adbbcbd96 None admin] RBAC: Authorizing `identity:get_user(user_id=a3f295c9d0b6422b9f50eac9eb3c9f15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:30.066941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4f6d9-954d-4ea5-89aa-c94adbbcbd96 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:30.068815 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 161/315] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:29 2022] GET /identity/v3/users/a3f295c9d0b6422b9f50eac9eb3c9f15 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:30.074462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261bfc7-2cda-40f9-b58e-32cec602fcb4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:30.124253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:30.182590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261bfc7-2cda-40f9-b58e-32cec602fcb4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:30.222157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f2c46-72e9-4274-a8d0-cae3b704bdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:30.222778 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f2c46-72e9-4274-a8d0-cae3b704bdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:30.223059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f2c46-72e9-4274-a8d0-cae3b704bdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:30.223333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f2c46-72e9-4274-a8d0-cae3b704bdc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:30.293292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f2c46-72e9-4274-a8d0-cae3b704bdc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:30.294742 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f2c46-72e9-4274-a8d0-cae3b704bdc4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:30.338853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:30.393014 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 155/316] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:30.398820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4f6d9-954d-4ea5-89aa-c94adbbcbd96 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:30.503055 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4f6d9-954d-4ea5-89aa-c94adbbcbd96 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:30.537825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75fe46-de81-4bb9-97d2-98c909713405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:30.538871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75fe46-de81-4bb9-97d2-98c909713405 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:30.539370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75fe46-de81-4bb9-97d2-98c909713405 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:30.539856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75fe46-de81-4bb9-97d2-98c909713405 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:30.541231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75fe46-de81-4bb9-97d2-98c909713405 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:30.542930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75fe46-de81-4bb9-97d2-98c909713405 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:30.545128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 162/317] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:30 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:30.550620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f2c46-72e9-4274-a8d0-cae3b704bdc4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:30.664201 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f2c46-72e9-4274-a8d0-cae3b704bdc4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:30.700553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee68464-8651-46f3-ba18-44b4124052e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:30.701166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee68464-8651-46f3-ba18-44b4124052e9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:30.701442 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee68464-8651-46f3-ba18-44b4124052e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:30.701714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee68464-8651-46f3-ba18-44b4124052e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:30.702503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee68464-8651-46f3-ba18-44b4124052e9 None admin] RBAC: Adding query filter params (role.id=e2331f0635214139bdac452081a3f310, scope.project.id=6a913e1d666748509269614e0c708389, user.id=a3f295c9d0b6422b9f50eac9eb3c9f15) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:30.702807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee68464-8651-46f3-ba18-44b4124052e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:30.704298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee68464-8651-46f3-ba18-44b4124052e9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:30.710786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 156/318] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:30 2022] GET /identity/v3/role_assignments?user.id=a3f295c9d0b6422b9f50eac9eb3c9f15&scope.project.id=6a913e1d666748509269614e0c708389&role.id=e2331f0635214139bdac452081a3f310 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:31.111925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151ea1f-1fbb-4a67-b501-d77264a0dcec None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.112464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151ea1f-1fbb-4a67-b501-d77264a0dcec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.112538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151ea1f-1fbb-4a67-b501-d77264a0dcec None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:31.114031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 163/319] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:31.119811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4d2e4-93e4-4312-9ab2-7d068761f6c0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.120147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4d2e4-93e4-4312-9ab2-7d068761f6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.120426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4d2e4-93e4-4312-9ab2-7d068761f6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:31.139157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-95d4d2e4-93e4-4312-9ab2-7d068761f6c0 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:31.140386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-95d4d2e4-93e4-4312-9ab2-7d068761f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:31.142559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:31.211414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 157/320] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:31.235017 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151ea1f-1fbb-4a67-b501-d77264a0dcec None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:31.290940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:31.339956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151ea1f-1fbb-4a67-b501-d77264a0dcec None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:31.375718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b3fd5-02db-45b3-9089-669aac568657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:31.376730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b3fd5-02db-45b3-9089-669aac568657 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.377247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b3fd5-02db-45b3-9089-669aac568657 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.377726 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b3fd5-02db-45b3-9089-669aac568657 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:31.388170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b3fd5-02db-45b3-9089-669aac568657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-601b3fd5-02db-45b3-9089-669aac568657 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:31.393336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:31.394199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:31.394985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:31.397930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 164/321] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:31.403702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4d2e4-93e4-4312-9ab2-7d068761f6c0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:31.504988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4d2e4-93e4-4312-9ab2-7d068761f6c0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:31.539890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3081bd4-39b8-41af-bf00-56a5e6571261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:31.540530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3081bd4-39b8-41af-bf00-56a5e6571261 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.540811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3081bd4-39b8-41af-bf00-56a5e6571261 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.541079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3081bd4-39b8-41af-bf00-56a5e6571261 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:31.541826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3081bd4-39b8-41af-bf00-56a5e6571261 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:31.542130 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3081bd4-39b8-41af-bf00-56a5e6571261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:31.543614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3081bd4-39b8-41af-bf00-56a5e6571261 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:31.548904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:31.553411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 158/322] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:31.558813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b3fd5-02db-45b3-9089-669aac568657 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:31.574532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a7133-60fe-4292-9859-7d6f2f371df8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.574848 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a7133-60fe-4292-9859-7d6f2f371df8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.575195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a7133-60fe-4292-9859-7d6f2f371df8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:31.576749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 159/323] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:23:31.580175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba3ebc-f1d0-41e2-84a2-34eec7f2add4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.580950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba3ebc-f1d0-41e2-84a2-34eec7f2add4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.581477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba3ebc-f1d0-41e2-84a2-34eec7f2add4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:31.582954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 160/324] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:31.584594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1658f-ddaf-4207-b659-97190dadfea8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.584919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1658f-ddaf-4207-b659-97190dadfea8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.585142 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1658f-ddaf-4207-b659-97190dadfea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:31.602890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-35c1658f-ddaf-4207-b659-97190dadfea8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:31.604130 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-35c1658f-ddaf-4207-b659-97190dadfea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:31.660431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b3fd5-02db-45b3-9089-669aac568657 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:31.669775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 161/325] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:31.670814 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af366668-d73c-44aa-a8c3-c495ad40f0c0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.671097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af366668-d73c-44aa-a8c3-c495ad40f0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.671369 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af366668-d73c-44aa-a8c3-c495ad40f0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:31.674687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:31.689804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-af366668-d73c-44aa-a8c3-c495ad40f0c0 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:31.691164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-af366668-d73c-44aa-a8c3-c495ad40f0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:31.696170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9aeb8-e400-4960-8268-12a4792ddfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:31.696809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9aeb8-e400-4960-8268-12a4792ddfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.697093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9aeb8-e400-4960-8268-12a4792ddfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.697370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9aeb8-e400-4960-8268-12a4792ddfb5 None admin] PATH_INFO: `/v3/users/081e1b7ff09444cdaed2b2d68248b779` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:31.698527 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9aeb8-e400-4960-8268-12a4792ddfb5 None admin] RBAC: Authorizing `identity:get_user(user_id=081e1b7ff09444cdaed2b2d68248b779)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:31.700108 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9aeb8-e400-4960-8268-12a4792ddfb5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:31.702120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 165/326] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:31 2022] GET /identity/v3/users/081e1b7ff09444cdaed2b2d68248b779 => generated 293 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:31.702874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9aeb8-e400-4960-8268-12a4792ddfb5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:31.760828 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 162/327] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:31.761580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af366668-d73c-44aa-a8c3-c495ad40f0c0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:31.805766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9aeb8-e400-4960-8268-12a4792ddfb5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:31.820167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:31.839205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648fc83-338d-47e5-afa1-f8e05a8fae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:31.839864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648fc83-338d-47e5-afa1-f8e05a8fae39 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.840175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648fc83-338d-47e5-afa1-f8e05a8fae39 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.840450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648fc83-338d-47e5-afa1-f8e05a8fae39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:31.841569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648fc83-338d-47e5-afa1-f8e05a8fae39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:31.843008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648fc83-338d-47e5-afa1-f8e05a8fae39 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:31.844596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 166/328] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:31.846465 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648fc83-338d-47e5-afa1-f8e05a8fae39 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:31.862667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af366668-d73c-44aa-a8c3-c495ad40f0c0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:31.894442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:31.896460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec6282-5c5c-4586-9b6d-baf092b2a07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:31.897078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec6282-5c5c-4586-9b6d-baf092b2a07e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.897359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec6282-5c5c-4586-9b6d-baf092b2a07e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.897632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec6282-5c5c-4586-9b6d-baf092b2a07e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:31.898436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec6282-5c5c-4586-9b6d-baf092b2a07e None admin] RBAC: Adding query filter params (role.id=525e499f44594268b4f362c62bd35e00, user.id=081e1b7ff09444cdaed2b2d68248b779, scope.system=all) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:31.898744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec6282-5c5c-4586-9b6d-baf092b2a07e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:31.900190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec6282-5c5c-4586-9b6d-baf092b2a07e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:31.912464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 163/329] 10.222.0.64 () {56 vars in 1308 bytes} [Thu Aug 4 07:23:31 2022] GET /identity/v3/role_assignments?user.id=081e1b7ff09444cdaed2b2d68248b779&scope.system=all&role.id=525e499f44594268b4f362c62bd35e00 => generated 510 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:31.913309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec6282-5c5c-4586-9b6d-baf092b2a07e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:31.953712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648fc83-338d-47e5-afa1-f8e05a8fae39 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:31.992703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5d2c4-4b65-47fa-8929-c4dfc752b659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:31.993374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5d2c4-4b65-47fa-8929-c4dfc752b659 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:31.993659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5d2c4-4b65-47fa-8929-c4dfc752b659 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:31.993936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5d2c4-4b65-47fa-8929-c4dfc752b659 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:32.006992 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5d2c4-4b65-47fa-8929-c4dfc752b659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-72b5d2c4-4b65-47fa-8929-c4dfc752b659 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:32.012795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:32.013873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:32.014664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 167/330] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:32.020111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5d2c4-4b65-47fa-8929-c4dfc752b659 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:32.054741 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec6282-5c5c-4586-9b6d-baf092b2a07e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:32.076515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:32.097125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ac059-a633-4b28-9b8a-a442ced4c72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:32.097809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ac059-a633-4b28-9b8a-a442ced4c72f None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:32.098097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ac059-a633-4b28-9b8a-a442ced4c72f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:32.098371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ac059-a633-4b28-9b8a-a442ced4c72f None admin] PATH_INFO: `/v3/groups` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:32.099171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ac059-a633-4b28-9b8a-a442ced4c72f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:32.100843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ac059-a633-4b28-9b8a-a442ced4c72f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:32.134234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5d2c4-4b65-47fa-8929-c4dfc752b659 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:32.143217 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:32.168387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cd7da-c64b-4d42-a6d7-ee595daea220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:32.169196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd7da-c64b-4d42-a6d7-ee595daea220 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:32.169483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd7da-c64b-4d42-a6d7-ee595daea220 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:32.169762 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd7da-c64b-4d42-a6d7-ee595daea220 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:32.170472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cd7da-c64b-4d42-a6d7-ee595daea220 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:32.170785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cd7da-c64b-4d42-a6d7-ee595daea220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:32.172193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cd7da-c64b-4d42-a6d7-ee595daea220 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:32.174914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 164/331] 10.222.0.64 () {60 vars in 1134 bytes} [Thu Aug 4 07:23:31 2022] POST /identity/v3/groups => generated 226 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:23:32.181561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 168/332] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:32.187420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ac059-a633-4b28-9b8a-a442ced4c72f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:32.377210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ac059-a633-4b28-9b8a-a442ced4c72f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:32.414944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada88d3-067d-4bf6-889b-737d4b7350cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:32.415594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada88d3-067d-4bf6-889b-737d4b7350cb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:32.415880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada88d3-067d-4bf6-889b-737d4b7350cb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:32.416187 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada88d3-067d-4bf6-889b-737d4b7350cb None admin] PATH_INFO: `/v3/users/5ec4a3dcb6804da8aa00ad2efa718450` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:32.417375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada88d3-067d-4bf6-889b-737d4b7350cb None admin] RBAC: Authorizing `identity:get_user(user_id=5ec4a3dcb6804da8aa00ad2efa718450)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:32.418820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada88d3-067d-4bf6-889b-737d4b7350cb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:32.420878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 165/333] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:32 2022] GET /identity/v3/users/5ec4a3dcb6804da8aa00ad2efa718450 => generated 293 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:32.426272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cd7da-c64b-4d42-a6d7-ee595daea220 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:32.535854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cd7da-c64b-4d42-a6d7-ee595daea220 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:32.572336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f10e2-ac97-427c-9ffb-78b394e5a969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:32.573008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f10e2-ac97-427c-9ffb-78b394e5a969 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:32.573302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f10e2-ac97-427c-9ffb-78b394e5a969 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:32.573565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f10e2-ac97-427c-9ffb-78b394e5a969 None admin] PATH_INFO: `/v3/system/users/5ec4a3dcb6804da8aa00ad2efa718450/roles/e2331f0635214139bdac452081a3f310` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:32.575034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f10e2-ac97-427c-9ffb-78b394e5a969 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5ec4a3dcb6804da8aa00ad2efa718450, role_id=e2331f0635214139bdac452081a3f310)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:32.576477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f10e2-ac97-427c-9ffb-78b394e5a969 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:32.645029 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 169/334] 10.222.0.64 () {58 vars in 1334 bytes} [Thu Aug 4 07:23:32 2022] PUT /identity/v3/system/users/5ec4a3dcb6804da8aa00ad2efa718450/roles/e2331f0635214139bdac452081a3f310 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:23:34.437853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71334bc-5389-4170-a4e0-032260969569 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:34.438909 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71334bc-5389-4170-a4e0-032260969569 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:34.439424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71334bc-5389-4170-a4e0-032260969569 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:34.441294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 166/335] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:34.446458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1bc17-5528-473d-b852-9c17f4f965c8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:34.447331 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1bc17-5528-473d-b852-9c17f4f965c8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:34.447821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1bc17-5528-473d-b852-9c17f4f965c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:34.472262 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e9f1bc17-5528-473d-b852-9c17f4f965c8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:34.474703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e9f1bc17-5528-473d-b852-9c17f4f965c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:34.506825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:34.561582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 170/336] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:34.601505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71334bc-5389-4170-a4e0-032260969569 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:34.660759 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:34.723259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71334bc-5389-4170-a4e0-032260969569 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:34.767513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77b384-64e7-4685-95c2-b2cc1567028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:34.768583 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77b384-64e7-4685-95c2-b2cc1567028b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:34.769086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77b384-64e7-4685-95c2-b2cc1567028b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:34.769561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77b384-64e7-4685-95c2-b2cc1567028b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:34.785035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77b384-64e7-4685-95c2-b2cc1567028b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-db77b384-64e7-4685-95c2-b2cc1567028b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:34.791346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:34.792308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:34.793123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:34.795748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 167/337] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:34.801569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1bc17-5528-473d-b852-9c17f4f965c8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:34.920187 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1bc17-5528-473d-b852-9c17f4f965c8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:34.964420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5b76e-18a5-4224-aea3-4dde31f1287f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:34.965053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b76e-18a5-4224-aea3-4dde31f1287f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:34.965337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b76e-18a5-4224-aea3-4dde31f1287f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:34.965612 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b76e-18a5-4224-aea3-4dde31f1287f None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:34.966337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5b76e-18a5-4224-aea3-4dde31f1287f None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:34.966641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5b76e-18a5-4224-aea3-4dde31f1287f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:34.968063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5b76e-18a5-4224-aea3-4dde31f1287f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:34.979777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 171/338] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:34.985351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77b384-64e7-4685-95c2-b2cc1567028b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:35.017778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14640783-9b94-4bb4-99fa-8f70af1f06f0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:35.018673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14640783-9b94-4bb4-99fa-8f70af1f06f0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:35.019163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14640783-9b94-4bb4-99fa-8f70af1f06f0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:35.021188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 172/339] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:23:35.034883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196578b-3ee8-477a-bc61-dd4d7417ff3e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:35.035191 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196578b-3ee8-477a-bc61-dd4d7417ff3e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:35.035443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196578b-3ee8-477a-bc61-dd4d7417ff3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:35.062939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a196578b-3ee8-477a-bc61-dd4d7417ff3e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:35.064161 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a196578b-3ee8-477a-bc61-dd4d7417ff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:35.100467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:35.139970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77b384-64e7-4685-95c2-b2cc1567028b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:35.148854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:35.168221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 173/340] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:35.169983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3900a-1576-48fb-9e29-e72ebb33caaf None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:35.170579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3900a-1576-48fb-9e29-e72ebb33caaf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:35.171343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3900a-1576-48fb-9e29-e72ebb33caaf None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:35.177931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 174/341] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:35.183847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a7d14-8a5f-41f3-aff6-848bab7c588c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:35.184643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a7d14-8a5f-41f3-aff6-848bab7c588c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:35.185360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a7d14-8a5f-41f3-aff6-848bab7c588c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:35.189885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51ad70-83b6-4c39-b850-77c18ffdfedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:35.190501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51ad70-83b6-4c39-b850-77c18ffdfedf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:35.190786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51ad70-83b6-4c39-b850-77c18ffdfedf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:35.191057 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51ad70-83b6-4c39-b850-77c18ffdfedf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:35.216977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7b0a7d14-8a5f-41f3-aff6-848bab7c588c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:35.218490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7b0a7d14-8a5f-41f3-aff6-848bab7c588c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:35.282312 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51ad70-83b6-4c39-b850-77c18ffdfedf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:35.283702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51ad70-83b6-4c39-b850-77c18ffdfedf None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:35.302496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 175/342] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:35.303822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a7d14-8a5f-41f3-aff6-848bab7c588c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:35.445268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:35.470589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 168/343] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 07:23:35.471454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51ad70-83b6-4c39-b850-77c18ffdfedf None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:35.476003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a7d14-8a5f-41f3-aff6-848bab7c588c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:35.500337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:35.549937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a6e50-0dd6-4790-8e37-6258c01f052f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:35.550564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a6e50-0dd6-4790-8e37-6258c01f052f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:35.550852 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a6e50-0dd6-4790-8e37-6258c01f052f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:35.551131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a6e50-0dd6-4790-8e37-6258c01f052f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:35.573694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a6e50-0dd6-4790-8e37-6258c01f052f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1f4a6e50-0dd6-4790-8e37-6258c01f052f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:35.578739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:35.579627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:35.580446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:35.581513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 176/344] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:35.586092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a6e50-0dd6-4790-8e37-6258c01f052f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:35.682286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51ad70-83b6-4c39-b850-77c18ffdfedf None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:35.784528 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272b846-c542-41ce-8978-dd511d3d97fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:35.785178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272b846-c542-41ce-8978-dd511d3d97fd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:35.785465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272b846-c542-41ce-8978-dd511d3d97fd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:35.788822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272b846-c542-41ce-8978-dd511d3d97fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:35.806475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272b846-c542-41ce-8978-dd511d3d97fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5272b846-c542-41ce-8978-dd511d3d97fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:35.811488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:35.812514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:35.813326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 169/345] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:35.813983 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272b846-c542-41ce-8978-dd511d3d97fd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:35.830757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a6e50-0dd6-4790-8e37-6258c01f052f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:35.868183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d11c2-f3a6-4bf2-aaa0-e2e5531bb55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:35.868838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d11c2-f3a6-4bf2-aaa0-e2e5531bb55d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:35.869126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d11c2-f3a6-4bf2-aaa0-e2e5531bb55d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:35.869405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d11c2-f3a6-4bf2-aaa0-e2e5531bb55d None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:35.870583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d11c2-f3a6-4bf2-aaa0-e2e5531bb55d None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:35.872175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d11c2-f3a6-4bf2-aaa0-e2e5531bb55d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:35.874013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 177/346] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:35 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:35.875071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ad69c-505f-4354-b9b2-d1b6ea1ec8c8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:35.875366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ad69c-505f-4354-b9b2-d1b6ea1ec8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:35.875645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ad69c-505f-4354-b9b2-d1b6ea1ec8c8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:35.876888 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 178/347] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:35.877576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ad69c-505f-4354-b9b2-d1b6ea1ec8c8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:35.888368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:35.900790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:35.926097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272b846-c542-41ce-8978-dd511d3d97fd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:35.962696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdb4e2-8841-46b4-93c9-b10606874c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:35.963318 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdb4e2-8841-46b4-93c9-b10606874c01 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:35.963597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdb4e2-8841-46b4-93c9-b10606874c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:35.963873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdb4e2-8841-46b4-93c9-b10606874c01 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:35.965148 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdb4e2-8841-46b4-93c9-b10606874c01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:35.965148 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdb4e2-8841-46b4-93c9-b10606874c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:35.966602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdb4e2-8841-46b4-93c9-b10606874c01 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:35.977232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 170/348] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:35.978147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdb4e2-8841-46b4-93c9-b10606874c01 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:36.001697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ad69c-505f-4354-b9b2-d1b6ea1ec8c8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:36.040045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5166b-b475-4b63-8b91-100beec3b5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:36.041007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5166b-b475-4b63-8b91-100beec3b5ee None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:36.041565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5166b-b475-4b63-8b91-100beec3b5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:36.042051 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5166b-b475-4b63-8b91-100beec3b5ee None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:36.043021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5166b-b475-4b63-8b91-100beec3b5ee None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:36.043545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5166b-b475-4b63-8b91-100beec3b5ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:36.045250 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5166b-b475-4b63-8b91-100beec3b5ee None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:36.056255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 179/349] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:36.057916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77786318-1886-4fe9-bf5e-8c7d3a75c565 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:36.058409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77786318-1886-4fe9-bf5e-8c7d3a75c565 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:36.058968 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77786318-1886-4fe9-bf5e-8c7d3a75c565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:36.078342 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-77786318-1886-4fe9-bf5e-8c7d3a75c565 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:36.079876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-77786318-1886-4fe9-bf5e-8c7d3a75c565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:36.085253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdb4e2-8841-46b4-93c9-b10606874c01 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:36.120451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:36.128056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf16d-807f-49f8-9429-c04c650fdfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:36.128056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf16d-807f-49f8-9429-c04c650fdfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:36.128296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf16d-807f-49f8-9429-c04c650fdfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:36.128519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf16d-807f-49f8-9429-c04c650fdfa9 None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:36.137088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aaf16d-807f-49f8-9429-c04c650fdfa9 None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:36.139352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aaf16d-807f-49f8-9429-c04c650fdfa9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:36.142637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 171/350] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:35 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:36.143605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf16d-807f-49f8-9429-c04c650fdfa9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:36.179456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 180/351] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:36.180424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77786318-1886-4fe9-bf5e-8c7d3a75c565 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:36.196197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:36.273438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf16d-807f-49f8-9429-c04c650fdfa9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:36.312850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6398e7-93b6-44ab-93cf-ae9bc02f089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:36.313712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6398e7-93b6-44ab-93cf-ae9bc02f089a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:36.314154 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6398e7-93b6-44ab-93cf-ae9bc02f089a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:36.314280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77786318-1886-4fe9-bf5e-8c7d3a75c565 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:36.314698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6398e7-93b6-44ab-93cf-ae9bc02f089a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:36.362750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f9348-dfcf-439a-a718-421e4c9003f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:36.363641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f9348-dfcf-439a-a718-421e4c9003f0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:36.364142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f9348-dfcf-439a-a718-421e4c9003f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:36.364590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f9348-dfcf-439a-a718-421e4c9003f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:36.386793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:36.397663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6398e7-93b6-44ab-93cf-ae9bc02f089a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:36.399138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6398e7-93b6-44ab-93cf-ae9bc02f089a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:36.451068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f9348-dfcf-439a-a718-421e4c9003f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:36.452627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f9348-dfcf-439a-a718-421e4c9003f0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:36.461447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:36.537713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 172/352] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:36.537713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6398e7-93b6-44ab-93cf-ae9bc02f089a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:36.585478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 181/353] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:36.586720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ccc0c-7423-4cf2-adb7-8c83a0a35b58 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:36.587075 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ccc0c-7423-4cf2-adb7-8c83a0a35b58 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:36.587477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ccc0c-7423-4cf2-adb7-8c83a0a35b58 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:36.589094 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 182/354] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:36.589796 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ccc0c-7423-4cf2-adb7-8c83a0a35b58 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:36.659278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:36.671891 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6398e7-93b6-44ab-93cf-ae9bc02f089a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:36.717955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c397393-6ba3-4d5d-a310-839295dba480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:36.718579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c397393-6ba3-4d5d-a310-839295dba480 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:36.718927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c397393-6ba3-4d5d-a310-839295dba480 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:36.719206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c397393-6ba3-4d5d-a310-839295dba480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:36.720019 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c397393-6ba3-4d5d-a310-839295dba480 None admin] RBAC: Adding query filter params (group.id=58bbc909690d4880b585e3eb7ae6efab, role.id=525e499f44594268b4f362c62bd35e00, scope.project.id=ce53e46475f341a2af8bf9da4700fffe) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:36.721047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c397393-6ba3-4d5d-a310-839295dba480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:36.722149 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c397393-6ba3-4d5d-a310-839295dba480 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:36.725125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ccc0c-7423-4cf2-adb7-8c83a0a35b58 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:36.730134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 173/355] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Aug 4 07:23:36 2022] GET /identity/v3/role_assignments?group.id=58bbc909690d4880b585e3eb7ae6efab&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=525e499f44594268b4f362c62bd35e00 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:36.730750 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c397393-6ba3-4d5d-a310-839295dba480 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:36.734590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:36.781577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951bb77-1ec7-4a4c-b04a-d9a1c49b98e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:36.782242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951bb77-1ec7-4a4c-b04a-d9a1c49b98e2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:36.782523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951bb77-1ec7-4a4c-b04a-d9a1c49b98e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:36.782798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951bb77-1ec7-4a4c-b04a-d9a1c49b98e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:36.796648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951bb77-1ec7-4a4c-b04a-d9a1c49b98e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7951bb77-1ec7-4a4c-b04a-d9a1c49b98e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:36.801922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:36.802914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:36.804110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 183/356] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:36.805110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17220198-f7cd-40a0-8985-9de7e0afe5b5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:36.805190 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17220198-f7cd-40a0-8985-9de7e0afe5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:36.806649 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17220198-f7cd-40a0-8985-9de7e0afe5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:36.826997 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-17220198-f7cd-40a0-8985-9de7e0afe5b5 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:36.828279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-17220198-f7cd-40a0-8985-9de7e0afe5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:36.876046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c397393-6ba3-4d5d-a310-839295dba480 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:36.920019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 184/357] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:36.920875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17220198-f7cd-40a0-8985-9de7e0afe5b5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:36.922243 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7374d7-cfa1-4dfc-8cff-1da81ccd600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:36.922894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7374d7-cfa1-4dfc-8cff-1da81ccd600c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:36.923231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7374d7-cfa1-4dfc-8cff-1da81ccd600c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:36.923505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7374d7-cfa1-4dfc-8cff-1da81ccd600c None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:36.924924 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7374d7-cfa1-4dfc-8cff-1da81ccd600c None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:36.926365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7374d7-cfa1-4dfc-8cff-1da81ccd600c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:36.928257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 174/358] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:36 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:36.934861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf07b50-f52d-4e84-83a3-82f087d59296 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:36.934861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf07b50-f52d-4e84-83a3-82f087d59296 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:36.934861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf07b50-f52d-4e84-83a3-82f087d59296 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:36.934861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 175/359] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:36.934861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf07b50-f52d-4e84-83a3-82f087d59296 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:36.962546 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:37.055325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:37.061476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf07b50-f52d-4e84-83a3-82f087d59296 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:37.066319 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17220198-f7cd-40a0-8985-9de7e0afe5b5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:37.106886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628c1ff-0dff-48a1-94c4-f242eff9ede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:37.107530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628c1ff-0dff-48a1-94c4-f242eff9ede7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:37.107810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628c1ff-0dff-48a1-94c4-f242eff9ede7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:37.108113 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628c1ff-0dff-48a1-94c4-f242eff9ede7 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:37.108331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192594dd-4a1b-43ae-bebf-91c7cc984799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:37.109489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192594dd-4a1b-43ae-bebf-91c7cc984799 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:37.109602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628c1ff-0dff-48a1-94c4-f242eff9ede7 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:37.110056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192594dd-4a1b-43ae-bebf-91c7cc984799 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:37.110479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192594dd-4a1b-43ae-bebf-91c7cc984799 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:37.110937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628c1ff-0dff-48a1-94c4-f242eff9ede7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:37.111517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192594dd-4a1b-43ae-bebf-91c7cc984799 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:37.111969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192594dd-4a1b-43ae-bebf-91c7cc984799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:37.112621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 185/360] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:36 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:37.113456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628c1ff-0dff-48a1-94c4-f242eff9ede7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:37.113815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192594dd-4a1b-43ae-bebf-91c7cc984799 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:37.126210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 176/361] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:37.128250 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54426d8f-7499-4d60-8393-df12126e7495 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:37.128749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54426d8f-7499-4d60-8393-df12126e7495 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:37.129224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54426d8f-7499-4d60-8393-df12126e7495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:37.151656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-54426d8f-7499-4d60-8393-df12126e7495 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:37.153212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-54426d8f-7499-4d60-8393-df12126e7495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:37.231259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 177/362] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:37.232144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54426d8f-7499-4d60-8393-df12126e7495 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:37.258282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:37.258478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628c1ff-0dff-48a1-94c4-f242eff9ede7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:37.298995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd52292-f0aa-4481-95b2-c5f1e2fb8c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:37.299846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd52292-f0aa-4481-95b2-c5f1e2fb8c53 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:37.299955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd52292-f0aa-4481-95b2-c5f1e2fb8c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:37.300216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd52292-f0aa-4481-95b2-c5f1e2fb8c53 None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:37.301456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd52292-f0aa-4481-95b2-c5f1e2fb8c53 None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:37.303035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd52292-f0aa-4481-95b2-c5f1e2fb8c53 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:37.304866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 186/363] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:37 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:37.305455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd52292-f0aa-4481-95b2-c5f1e2fb8c53 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:37.351007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:37.388526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54426d8f-7499-4d60-8393-df12126e7495 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:37.436018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953c330-3824-4cb7-b2f3-5fe8ccd228e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:37.436682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953c330-3824-4cb7-b2f3-5fe8ccd228e3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:37.436959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953c330-3824-4cb7-b2f3-5fe8ccd228e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:37.437235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953c330-3824-4cb7-b2f3-5fe8ccd228e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:37.440442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd52292-f0aa-4481-95b2-c5f1e2fb8c53 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:37.449640 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953c330-3824-4cb7-b2f3-5fe8ccd228e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1953c330-3824-4cb7-b2f3-5fe8ccd228e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:37.456802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:37.457753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:37.459091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:37.461620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 178/364] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:37.462570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953c330-3824-4cb7-b2f3-5fe8ccd228e3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:37.488190 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103d590-6801-4169-b5d3-09d66b948cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:37.489182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103d590-6801-4169-b5d3-09d66b948cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:37.489750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103d590-6801-4169-b5d3-09d66b948cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:37.490226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103d590-6801-4169-b5d3-09d66b948cd4 None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:37.492258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103d590-6801-4169-b5d3-09d66b948cd4 None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:37.493903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103d590-6801-4169-b5d3-09d66b948cd4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:37.495685 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 187/365] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:37 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:37.496659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103d590-6801-4169-b5d3-09d66b948cd4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:37.580278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:37.610297 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953c330-3824-4cb7-b2f3-5fe8ccd228e3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:37.630616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:37.634517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103d590-6801-4169-b5d3-09d66b948cd4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:37.656076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa5a08-d115-4ce2-830f-deeabef7b9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:37.656649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa5a08-d115-4ce2-830f-deeabef7b9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:37.657376 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa5a08-d115-4ce2-830f-deeabef7b9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:37.657661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa5a08-d115-4ce2-830f-deeabef7b9a6 None admin] PATH_INFO: `/v3/users/5ec4a3dcb6804da8aa00ad2efa718450` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:37.658864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa5a08-d115-4ce2-830f-deeabef7b9a6 None admin] RBAC: Authorizing `identity:get_user(user_id=5ec4a3dcb6804da8aa00ad2efa718450)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:37.661887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa5a08-d115-4ce2-830f-deeabef7b9a6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:37.667769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 179/366] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:23:37 2022] GET /identity/v3/users/5ec4a3dcb6804da8aa00ad2efa718450 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:37.669524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa5a08-d115-4ce2-830f-deeabef7b9a6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:37.679201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159b0e1-7bef-484e-b0f6-bce82d67899c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:37.679846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159b0e1-7bef-484e-b0f6-bce82d67899c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:37.680157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159b0e1-7bef-484e-b0f6-bce82d67899c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:37.680447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159b0e1-7bef-484e-b0f6-bce82d67899c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:37.692400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159b0e1-7bef-484e-b0f6-bce82d67899c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1159b0e1-7bef-484e-b0f6-bce82d67899c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:37.697638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:37.698535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:37.699348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 188/367] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:37.700094 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159b0e1-7bef-484e-b0f6-bce82d67899c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:37.784244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa5a08-d115-4ce2-830f-deeabef7b9a6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:37.831023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513463e9-0499-4a91-9eac-aa800a8943bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:37.831648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513463e9-0499-4a91-9eac-aa800a8943bd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:37.831928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513463e9-0499-4a91-9eac-aa800a8943bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:37.832229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513463e9-0499-4a91-9eac-aa800a8943bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:37.833238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513463e9-0499-4a91-9eac-aa800a8943bd None admin] RBAC: Adding query filter params (group.id=58bbc909690d4880b585e3eb7ae6efab, role.id=e3c5a858a45c41db8ff3e883788d75bc, scope.project.id=ce53e46475f341a2af8bf9da4700fffe) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:37.833562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513463e9-0499-4a91-9eac-aa800a8943bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:37.835074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513463e9-0499-4a91-9eac-aa800a8943bd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:37.836644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159b0e1-7bef-484e-b0f6-bce82d67899c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:37.848955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 180/368] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Aug 4 07:23:37 2022] GET /identity/v3/role_assignments?group.id=58bbc909690d4880b585e3eb7ae6efab&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=e3c5a858a45c41db8ff3e883788d75bc => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:37.850926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513463e9-0499-4a91-9eac-aa800a8943bd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:37.868414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:37.885178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466878c9-fbc3-4c8f-b022-306b630206de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:37.886141 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466878c9-fbc3-4c8f-b022-306b630206de None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:37.888202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466878c9-fbc3-4c8f-b022-306b630206de None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:37.888202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466878c9-fbc3-4c8f-b022-306b630206de None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:37.888202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466878c9-fbc3-4c8f-b022-306b630206de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:37.888202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466878c9-fbc3-4c8f-b022-306b630206de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:37.890054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466878c9-fbc3-4c8f-b022-306b630206de None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:37.900947 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 189/369] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:37.902246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466878c9-fbc3-4c8f-b022-306b630206de None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:37.937281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:38.004564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513463e9-0499-4a91-9eac-aa800a8943bd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:38.047789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc56878-ba40-4526-a3b6-814db0c7fbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:38.048551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc56878-ba40-4526-a3b6-814db0c7fbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:38.048829 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466878c9-fbc3-4c8f-b022-306b630206de None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:38.048949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc56878-ba40-4526-a3b6-814db0c7fbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:38.049156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc56878-ba40-4526-a3b6-814db0c7fbb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:38.049911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc56878-ba40-4526-a3b6-814db0c7fbb5 None admin] RBAC: Adding query filter params (group.id=58bbc909690d4880b585e3eb7ae6efab, role.id=525e499f44594268b4f362c62bd35e00, scope.project.id=6a913e1d666748509269614e0c708389) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:38.050212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc56878-ba40-4526-a3b6-814db0c7fbb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:38.052631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc56878-ba40-4526-a3b6-814db0c7fbb5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:38.059937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 181/370] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Aug 4 07:23:37 2022] GET /identity/v3/role_assignments?group.id=58bbc909690d4880b585e3eb7ae6efab&scope.project.id=6a913e1d666748509269614e0c708389&role.id=525e499f44594268b4f362c62bd35e00 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:38.060836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc56878-ba40-4526-a3b6-814db0c7fbb5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:38.126053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a58fd-6e96-4443-b217-b85f15691182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:38.126692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a58fd-6e96-4443-b217-b85f15691182 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:38.126973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a58fd-6e96-4443-b217-b85f15691182 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:38.127247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a58fd-6e96-4443-b217-b85f15691182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:38.128078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a58fd-6e96-4443-b217-b85f15691182 None admin] RBAC: Adding query filter params (role.id=e2331f0635214139bdac452081a3f310, user.id=5ec4a3dcb6804da8aa00ad2efa718450, scope.system=all) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:38.128390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a58fd-6e96-4443-b217-b85f15691182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:38.130597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a58fd-6e96-4443-b217-b85f15691182 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:38.153719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 190/371] 10.222.0.64 () {56 vars in 1308 bytes} [Thu Aug 4 07:23:37 2022] GET /identity/v3/role_assignments?user.id=5ec4a3dcb6804da8aa00ad2efa718450&scope.system=all&role.id=e2331f0635214139bdac452081a3f310 => generated 510 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:38.154405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a58fd-6e96-4443-b217-b85f15691182 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:38.213506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:38.286489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc56878-ba40-4526-a3b6-814db0c7fbb5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:38.299353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:38.326148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a58fd-6e96-4443-b217-b85f15691182 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:38.372164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e325b0-7cc3-4e31-8423-aaae0b2729ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:38.372800 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e325b0-7cc3-4e31-8423-aaae0b2729ad None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:38.373080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e325b0-7cc3-4e31-8423-aaae0b2729ad None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:38.373349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e325b0-7cc3-4e31-8423-aaae0b2729ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:38.391284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43081af-b98a-435a-9615-8ab211f6fc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:38.391284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43081af-b98a-435a-9615-8ab211f6fc27 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:38.391284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43081af-b98a-435a-9615-8ab211f6fc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:38.391284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43081af-b98a-435a-9615-8ab211f6fc27 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:38.391805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43081af-b98a-435a-9615-8ab211f6fc27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:38.392144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43081af-b98a-435a-9615-8ab211f6fc27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:38.393515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43081af-b98a-435a-9615-8ab211f6fc27 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:38.419833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 182/372] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:38.425413 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43081af-b98a-435a-9615-8ab211f6fc27 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:38.452126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e325b0-7cc3-4e31-8423-aaae0b2729ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:38.453597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e325b0-7cc3-4e31-8423-aaae0b2729ad None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:38.552698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:38.576315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43081af-b98a-435a-9615-8ab211f6fc27 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:38.579853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 191/373] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:38.586263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e325b0-7cc3-4e31-8423-aaae0b2729ad None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:38.625763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c1016-9d3a-4ff5-af41-8714c3ce81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:38.626403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c1016-9d3a-4ff5-af41-8714c3ce81de None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:38.626661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c1016-9d3a-4ff5-af41-8714c3ce81de None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:38.627217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c1016-9d3a-4ff5-af41-8714c3ce81de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:38.632448 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:38.756203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c1016-9d3a-4ff5-af41-8714c3ce81de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:38.757733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c1016-9d3a-4ff5-af41-8714c3ce81de None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:38.765718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e325b0-7cc3-4e31-8423-aaae0b2729ad None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:38.836352 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde4e7a-0802-4a65-8d2b-6c1f49222b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:38.836980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde4e7a-0802-4a65-8d2b-6c1f49222b14 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:38.837262 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde4e7a-0802-4a65-8d2b-6c1f49222b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:38.837534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde4e7a-0802-4a65-8d2b-6c1f49222b14 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:38.839159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde4e7a-0802-4a65-8d2b-6c1f49222b14 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:38.840641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde4e7a-0802-4a65-8d2b-6c1f49222b14 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:38.846951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 192/374] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:38 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:38.847823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde4e7a-0802-4a65-8d2b-6c1f49222b14 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:38.949354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:38.964745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 183/375] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:38.969749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c1016-9d3a-4ff5-af41-8714c3ce81de None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:39.028372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:39.066650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde4e7a-0802-4a65-8d2b-6c1f49222b14 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:39.111243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c47643-dc11-469b-acf7-9914c4d119dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:39.112010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c47643-dc11-469b-acf7-9914c4d119dc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:39.112411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c47643-dc11-469b-acf7-9914c4d119dc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:39.112765 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c47643-dc11-469b-acf7-9914c4d119dc None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:39.114937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c47643-dc11-469b-acf7-9914c4d119dc None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:39.116513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c47643-dc11-469b-acf7-9914c4d119dc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:39.118444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c1016-9d3a-4ff5-af41-8714c3ce81de None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:39.119263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 193/376] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:38 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:39.120792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203fba6-3224-44e8-8b44-d82512e09497 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:39.121177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203fba6-3224-44e8-8b44-d82512e09497 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:39.121711 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203fba6-3224-44e8-8b44-d82512e09497 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:39.124876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 194/377] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:39.133052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca71b2-3482-4e8c-bced-ab5ade31d8ea None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:39.134628 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca71b2-3482-4e8c-bced-ab5ade31d8ea None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:39.136183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca71b2-3482-4e8c-bced-ab5ade31d8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:39.163429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e9ca71b2-3482-4e8c-bced-ab5ade31d8ea None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:39.164844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e9ca71b2-3482-4e8c-bced-ab5ade31d8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:39.171164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918a077-922e-4139-b745-ba5c3ac3d3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:39.173692 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918a077-922e-4139-b745-ba5c3ac3d3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:39.174428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918a077-922e-4139-b745-ba5c3ac3d3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:39.174914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918a077-922e-4139-b745-ba5c3ac3d3a5 None admin] PATH_INFO: `/v3/projects/60367118cabe491a9e4ea2e0f6945aa3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:39.176311 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918a077-922e-4139-b745-ba5c3ac3d3a5 None admin] RBAC: Authorizing `identity:get_project(project_id=60367118cabe491a9e4ea2e0f6945aa3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:39.178820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918a077-922e-4139-b745-ba5c3ac3d3a5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:39.182674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 184/378] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:38 2022] GET /identity/v3/projects/60367118cabe491a9e4ea2e0f6945aa3 => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:39.185598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918a077-922e-4139-b745-ba5c3ac3d3a5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:39.254158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 195/379] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:39.255924 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca71b2-3482-4e8c-bced-ab5ade31d8ea None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:39.310318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:39.374615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca71b2-3482-4e8c-bced-ab5ade31d8ea None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:39.376143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918a077-922e-4139-b745-ba5c3ac3d3a5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:39.396655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:39.419614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d624779-5467-421f-a181-7e1bf2dc1017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:39.420400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d624779-5467-421f-a181-7e1bf2dc1017 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:39.420730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d624779-5467-421f-a181-7e1bf2dc1017 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:39.421010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d624779-5467-421f-a181-7e1bf2dc1017 None admin] PATH_INFO: `/v3/groups/179572c9f41d4b819119233e892d2344` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:39.428539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d624779-5467-421f-a181-7e1bf2dc1017 None admin] RBAC: Authorizing `identity:get_group(group_id=179572c9f41d4b819119233e892d2344)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:39.429993 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d624779-5467-421f-a181-7e1bf2dc1017 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:39.431637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 196/380] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:39 2022] GET /identity/v3/groups/179572c9f41d4b819119233e892d2344 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:39.432494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d624779-5467-421f-a181-7e1bf2dc1017 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:39.433354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2f90a-8f91-43c6-bdf6-a3ea78ff86bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:39.433976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2f90a-8f91-43c6-bdf6-a3ea78ff86bd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:39.434262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2f90a-8f91-43c6-bdf6-a3ea78ff86bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:39.434542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2f90a-8f91-43c6-bdf6-a3ea78ff86bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:39.435340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2f90a-8f91-43c6-bdf6-a3ea78ff86bd None admin] RBAC: Adding query filter params (group.id=58bbc909690d4880b585e3eb7ae6efab, role.id=e3c5a858a45c41db8ff3e883788d75bc, scope.project.id=6a913e1d666748509269614e0c708389) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:39.435652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2f90a-8f91-43c6-bdf6-a3ea78ff86bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:39.439504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2f90a-8f91-43c6-bdf6-a3ea78ff86bd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:39.453062 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 185/381] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Aug 4 07:23:39 2022] GET /identity/v3/role_assignments?group.id=58bbc909690d4880b585e3eb7ae6efab&scope.project.id=6a913e1d666748509269614e0c708389&role.id=e3c5a858a45c41db8ff3e883788d75bc => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:39.454115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2f90a-8f91-43c6-bdf6-a3ea78ff86bd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:39.567900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2f90a-8f91-43c6-bdf6-a3ea78ff86bd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:39.575616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d624779-5467-421f-a181-7e1bf2dc1017 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:39.590165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:39.619349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8361ca-c0aa-43b0-8e43-e9cba0058603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:39.620118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8361ca-c0aa-43b0-8e43-e9cba0058603 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:39.620418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8361ca-c0aa-43b0-8e43-e9cba0058603 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:39.620679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8361ca-c0aa-43b0-8e43-e9cba0058603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:39.624006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8361ca-c0aa-43b0-8e43-e9cba0058603 None admin] RBAC: Adding query filter params (group.id=179572c9f41d4b819119233e892d2344, role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=60367118cabe491a9e4ea2e0f6945aa3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:39.624006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8361ca-c0aa-43b0-8e43-e9cba0058603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:39.624006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8361ca-c0aa-43b0-8e43-e9cba0058603 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:39.642727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 186/382] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Aug 4 07:23:39 2022] GET /identity/v3/role_assignments?group.id=179572c9f41d4b819119233e892d2344&scope.project.id=60367118cabe491a9e4ea2e0f6945aa3&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:39.653697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0812e0-c694-4a28-a932-9b65bae44fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:39.654363 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0812e0-c694-4a28-a932-9b65bae44fde None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:39.654643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0812e0-c694-4a28-a932-9b65bae44fde None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:39.654918 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0812e0-c694-4a28-a932-9b65bae44fde None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:39.673093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0812e0-c694-4a28-a932-9b65bae44fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8d0812e0-c694-4a28-a932-9b65bae44fde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:39.679517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:39.680570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:39.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 197/383] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:39.688096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8361ca-c0aa-43b0-8e43-e9cba0058603 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:39.735792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:39.836985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8361ca-c0aa-43b0-8e43-e9cba0058603 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:39.893285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ace5d-049f-485e-8df9-2e9afa11373e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:39.893836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ace5d-049f-485e-8df9-2e9afa11373e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:39.894119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ace5d-049f-485e-8df9-2e9afa11373e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:39.894394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ace5d-049f-485e-8df9-2e9afa11373e None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:39.895334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ace5d-049f-485e-8df9-2e9afa11373e None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:39.896354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ace5d-049f-485e-8df9-2e9afa11373e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:39.898780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ace5d-049f-485e-8df9-2e9afa11373e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:39.920130 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 187/384] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:39.922732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0812e0-c694-4a28-a932-9b65bae44fde None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:40.069825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0812e0-c694-4a28-a932-9b65bae44fde None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:40.115994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab15ccf-fe88-497e-9a12-5f81c578683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:40.116705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab15ccf-fe88-497e-9a12-5f81c578683d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:40.116988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab15ccf-fe88-497e-9a12-5f81c578683d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:40.117259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab15ccf-fe88-497e-9a12-5f81c578683d None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:40.118390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab15ccf-fe88-497e-9a12-5f81c578683d None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:40.119945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab15ccf-fe88-497e-9a12-5f81c578683d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:40.121613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 198/385] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:39 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:40.128518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ace5d-049f-485e-8df9-2e9afa11373e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:40.247780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:40.259779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ace5d-049f-485e-8df9-2e9afa11373e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:40.297731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8dc34-ba32-4e06-96f3-712a3e0bbf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:40.298379 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8dc34-ba32-4e06-96f3-712a3e0bbf23 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:40.298659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8dc34-ba32-4e06-96f3-712a3e0bbf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:40.299048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8dc34-ba32-4e06-96f3-712a3e0bbf23 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:40.301078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8dc34-ba32-4e06-96f3-712a3e0bbf23 None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:40.303634 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8dc34-ba32-4e06-96f3-712a3e0bbf23 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:40.306295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 188/386] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:40 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:40.315190 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab15ccf-fe88-497e-9a12-5f81c578683d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:40.377303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:40.470967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab15ccf-fe88-497e-9a12-5f81c578683d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:40.519408 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d699d39-98e2-4f89-943a-215f76e06eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:40.520027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699d39-98e2-4f89-943a-215f76e06eff None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:40.520388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699d39-98e2-4f89-943a-215f76e06eff None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:40.520839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699d39-98e2-4f89-943a-215f76e06eff None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe/groups/58bbc909690d4880b585e3eb7ae6efab/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:40.523420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d699d39-98e2-4f89-943a-215f76e06eff None admin] RBAC: Authorizing `identity:create_grant(project_id=ce53e46475f341a2af8bf9da4700fffe, group_id=58bbc909690d4880b585e3eb7ae6efab, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:40.525514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d699d39-98e2-4f89-943a-215f76e06eff None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:40.599364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 199/387] 10.222.0.64 () {58 vars in 1442 bytes} [Thu Aug 4 07:23:40 2022] PUT /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe/groups/58bbc909690d4880b585e3eb7ae6efab/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:40.821496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568033b3-7439-4c05-a917-fdda639aa08a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:40.821987 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568033b3-7439-4c05-a917-fdda639aa08a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:40.821987 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568033b3-7439-4c05-a917-fdda639aa08a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:40.823309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 189/388] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:40.841775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42a88f-28ca-4d32-aac2-76515f3a408d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:40.842079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42a88f-28ca-4d32-aac2-76515f3a408d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:40.842332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42a88f-28ca-4d32-aac2-76515f3a408d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:40.867422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cd42a88f-28ca-4d32-aac2-76515f3a408d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:40.867603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e9a50-a6a5-4627-954c-a148a9bb5134 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:40.867603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e9a50-a6a5-4627-954c-a148a9bb5134 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:40.867603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e9a50-a6a5-4627-954c-a148a9bb5134 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:40.867827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cd42a88f-28ca-4d32-aac2-76515f3a408d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:40.867895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 190/389] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:40.873473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10de4fd-d22a-4d5d-a4c5-cac837951d54 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:40.873759 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10de4fd-d22a-4d5d-a4c5-cac837951d54 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:40.874038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10de4fd-d22a-4d5d-a4c5-cac837951d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:40.904697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c10de4fd-d22a-4d5d-a4c5-cac837951d54 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:40.905925 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c10de4fd-d22a-4d5d-a4c5-cac837951d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:40.943942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 200/390] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:40.981464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42a88f-28ca-4d32-aac2-76515f3a408d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:40.991888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 191/391] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:41.017709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:41.034920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10de4fd-d22a-4d5d-a4c5-cac837951d54 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:41.101039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42a88f-28ca-4d32-aac2-76515f3a408d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:41.167513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fa3d4-852d-4a58-90b5-4cea8ad16eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:41.168194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fa3d4-852d-4a58-90b5-4cea8ad16eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:41.168482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fa3d4-852d-4a58-90b5-4cea8ad16eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:41.168763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fa3d4-852d-4a58-90b5-4cea8ad16eb3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:41.188939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fa3d4-852d-4a58-90b5-4cea8ad16eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e46fa3d4-852d-4a58-90b5-4cea8ad16eb3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:41.196793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:41.197666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:41.198476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:41.204369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 201/392] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:41.210609 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fa3d4-852d-4a58-90b5-4cea8ad16eb3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:41.210725 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:41.225826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10de4fd-d22a-4d5d-a4c5-cac837951d54 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:41.289738 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a28ad-5d28-4b17-b4ea-f896cf1b38ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:41.290393 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a28ad-5d28-4b17-b4ea-f896cf1b38ca None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:41.290677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a28ad-5d28-4b17-b4ea-f896cf1b38ca None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:41.290957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a28ad-5d28-4b17-b4ea-f896cf1b38ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:41.312985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a28ad-5d28-4b17-b4ea-f896cf1b38ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-819a28ad-5d28-4b17-b4ea-f896cf1b38ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:41.318631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:41.319515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:41.320329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 192/393] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:41.331977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a28ad-5d28-4b17-b4ea-f896cf1b38ca None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:41.369546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:41.384384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fa3d4-852d-4a58-90b5-4cea8ad16eb3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:41.429439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6788a-a4dc-4cdc-b6c1-b318ab9a0178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:41.430067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6788a-a4dc-4cdc-b6c1-b318ab9a0178 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:41.430354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6788a-a4dc-4cdc-b6c1-b318ab9a0178 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:41.430628 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6788a-a4dc-4cdc-b6c1-b318ab9a0178 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:41.431351 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6788a-a4dc-4cdc-b6c1-b318ab9a0178 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:41.432251 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6788a-a4dc-4cdc-b6c1-b318ab9a0178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:41.433646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6788a-a4dc-4cdc-b6c1-b318ab9a0178 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:41.444403 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 202/394] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:41.452823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6788a-a4dc-4cdc-b6c1-b318ab9a0178 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:41.558081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a28ad-5d28-4b17-b4ea-f896cf1b38ca None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:41.630620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ba4cb-2451-4f17-b86f-ce5d2452592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:41.631378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ba4cb-2451-4f17-b86f-ce5d2452592f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:41.631665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ba4cb-2451-4f17-b86f-ce5d2452592f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:41.631946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ba4cb-2451-4f17-b86f-ce5d2452592f None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:41.632699 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ba4cb-2451-4f17-b86f-ce5d2452592f None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:41.633010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ba4cb-2451-4f17-b86f-ce5d2452592f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:41.634399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ba4cb-2451-4f17-b86f-ce5d2452592f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:41.650755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6788a-a4dc-4cdc-b6c1-b318ab9a0178 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:41.654240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 193/395] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:41.660355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ba4cb-2451-4f17-b86f-ce5d2452592f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:41.682993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:41.732825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba9162-0b92-4277-8ad0-1a9060fda58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:41.733476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba9162-0b92-4277-8ad0-1a9060fda58a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:41.733993 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba9162-0b92-4277-8ad0-1a9060fda58a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:41.734276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba9162-0b92-4277-8ad0-1a9060fda58a None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:41.736073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba9162-0b92-4277-8ad0-1a9060fda58a None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:41.737529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba9162-0b92-4277-8ad0-1a9060fda58a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:41.740197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 203/396] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:41 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:41.746037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba9162-0b92-4277-8ad0-1a9060fda58a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:41.780348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:41.837375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ba4cb-2451-4f17-b86f-ce5d2452592f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:41.900752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd997ce-a41e-4de3-8815-87155c03d6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:41.901314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd997ce-a41e-4de3-8815-87155c03d6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:41.901594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd997ce-a41e-4de3-8815-87155c03d6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:41.901864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd997ce-a41e-4de3-8815-87155c03d6a6 None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:41.905285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd997ce-a41e-4de3-8815-87155c03d6a6 None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:41.906714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd997ce-a41e-4de3-8815-87155c03d6a6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:41.909712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 194/397] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:41 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:41.921061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd997ce-a41e-4de3-8815-87155c03d6a6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:41.928496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba9162-0b92-4277-8ad0-1a9060fda58a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:41.968678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82843bb7-2797-4ec0-95fc-715bd89458d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:41.970076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82843bb7-2797-4ec0-95fc-715bd89458d6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:41.970560 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82843bb7-2797-4ec0-95fc-715bd89458d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:41.971061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82843bb7-2797-4ec0-95fc-715bd89458d6 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:41.972689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82843bb7-2797-4ec0-95fc-715bd89458d6 None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:41.974968 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82843bb7-2797-4ec0-95fc-715bd89458d6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:41.977092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 204/398] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:41 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:41.978539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6bd53-c57a-41fc-aef3-9554fb12e5ca None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:41.978889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6bd53-c57a-41fc-aef3-9554fb12e5ca None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:41.979239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6bd53-c57a-41fc-aef3-9554fb12e5ca None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:41.984133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 205/399] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:41.986083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6bd53-c57a-41fc-aef3-9554fb12e5ca None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:42.032398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:42.064031 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd997ce-a41e-4de3-8815-87155c03d6a6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:42.082918 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:42.100164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6bd53-c57a-41fc-aef3-9554fb12e5ca None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:42.109547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acdebc-6538-4f6a-a531-62608e967911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:42.110201 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acdebc-6538-4f6a-a531-62608e967911 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:42.110678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acdebc-6538-4f6a-a531-62608e967911 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:42.111459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acdebc-6538-4f6a-a531-62608e967911 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:42.113504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acdebc-6538-4f6a-a531-62608e967911 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:42.115481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acdebc-6538-4f6a-a531-62608e967911 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:42.118879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 195/400] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:41 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:42.120324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001916a-f181-42d3-a0ba-b240cbf78328 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:42.120782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001916a-f181-42d3-a0ba-b240cbf78328 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:42.121206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001916a-f181-42d3-a0ba-b240cbf78328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:42.146216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5001916a-f181-42d3-a0ba-b240cbf78328 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:42.147658 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5001916a-f181-42d3-a0ba-b240cbf78328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:42.149026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a55ab-515e-432b-b983-b0aaa81ede7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:42.150072 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a55ab-515e-432b-b983-b0aaa81ede7e None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:42.150495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a55ab-515e-432b-b983-b0aaa81ede7e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:42.151795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a55ab-515e-432b-b983-b0aaa81ede7e None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe/groups/58bbc909690d4880b585e3eb7ae6efab/roles/e3c5a858a45c41db8ff3e883788d75bc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:42.154302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a55ab-515e-432b-b983-b0aaa81ede7e None admin] RBAC: Authorizing `identity:create_grant(project_id=ce53e46475f341a2af8bf9da4700fffe, group_id=58bbc909690d4880b585e3eb7ae6efab, role_id=e3c5a858a45c41db8ff3e883788d75bc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:42.156389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a55ab-515e-432b-b983-b0aaa81ede7e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:42.221588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 196/401] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:42.222451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 206/402] 10.222.0.64 () {58 vars in 1442 bytes} [Thu Aug 4 07:23:41 2022] PUT /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe/groups/58bbc909690d4880b585e3eb7ae6efab/roles/e3c5a858a45c41db8ff3e883788d75bc => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:42.222582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001916a-f181-42d3-a0ba-b240cbf78328 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:42.243928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a55ab-515e-432b-b983-b0aaa81ede7e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:42.318405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:42.343356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001916a-f181-42d3-a0ba-b240cbf78328 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:42.383505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1c021-68d7-4314-9038-881db35869a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:42.384533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1c021-68d7-4314-9038-881db35869a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:42.385524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1c021-68d7-4314-9038-881db35869a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:42.385958 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1c021-68d7-4314-9038-881db35869a9 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389/groups/58bbc909690d4880b585e3eb7ae6efab/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:42.388360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1c021-68d7-4314-9038-881db35869a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a913e1d666748509269614e0c708389, group_id=58bbc909690d4880b585e3eb7ae6efab, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:42.390115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1c021-68d7-4314-9038-881db35869a9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:42.435515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a55ab-515e-432b-b983-b0aaa81ede7e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:42.474458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4edf79-e94f-4bf6-815a-6c519ba10a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:42.475194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4edf79-e94f-4bf6-815a-6c519ba10a86 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:42.475482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4edf79-e94f-4bf6-815a-6c519ba10a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:42.475758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4edf79-e94f-4bf6-815a-6c519ba10a86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:42.486532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4edf79-e94f-4bf6-815a-6c519ba10a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bb4edf79-e94f-4bf6-815a-6c519ba10a86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:42.492303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:42.493193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:42.493965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 207/403] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:42.495131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082a18-3cc8-480b-8d92-2080ed69c745 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:42.495291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082a18-3cc8-480b-8d92-2080ed69c745 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:42.496014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082a18-3cc8-480b-8d92-2080ed69c745 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:42.500464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 208/404] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:42.501468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a082a18-3cc8-480b-8d92-2080ed69c745 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:42.512313 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:42.520608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 197/405] 10.222.0.64 () {58 vars in 1442 bytes} [Thu Aug 4 07:23:42 2022] PUT /identity/v3/projects/6a913e1d666748509269614e0c708389/groups/58bbc909690d4880b585e3eb7ae6efab/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:42.525701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f06f6d-5c4f-4685-a0c0-31e92f320f80 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:42.525701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f06f6d-5c4f-4685-a0c0-31e92f320f80 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:42.525701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f06f6d-5c4f-4685-a0c0-31e92f320f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:42.546241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-70f06f6d-5c4f-4685-a0c0-31e92f320f80 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:42.547877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-70f06f6d-5c4f-4685-a0c0-31e92f320f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:42.637351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 198/406] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:42.662608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f06f6d-5c4f-4685-a0c0-31e92f320f80 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:42.748506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:42.757325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a082a18-3cc8-480b-8d92-2080ed69c745 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:42.784897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f06f6d-5c4f-4685-a0c0-31e92f320f80 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:42.805379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c75d7e-e5ff-4fad-8054-7b8ecac7a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:42.806104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c75d7e-e5ff-4fad-8054-7b8ecac7a16d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:42.806395 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c75d7e-e5ff-4fad-8054-7b8ecac7a16d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:42.806675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c75d7e-e5ff-4fad-8054-7b8ecac7a16d None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:42.807401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c75d7e-e5ff-4fad-8054-7b8ecac7a16d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:42.807710 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c75d7e-e5ff-4fad-8054-7b8ecac7a16d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:42.809323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c75d7e-e5ff-4fad-8054-7b8ecac7a16d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:42.820722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 209/407] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:42.825063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c75d7e-e5ff-4fad-8054-7b8ecac7a16d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:42.849739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d652ae0-fd09-48db-b6a7-f9fdda42ef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:42.850398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d652ae0-fd09-48db-b6a7-f9fdda42ef6e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:42.850688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d652ae0-fd09-48db-b6a7-f9fdda42ef6e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:42.850965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d652ae0-fd09-48db-b6a7-f9fdda42ef6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:42.862513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d652ae0-fd09-48db-b6a7-f9fdda42ef6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3d652ae0-fd09-48db-b6a7-f9fdda42ef6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:42.868279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:42.869132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:42.869906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 199/408] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:23:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:42.880968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d652ae0-fd09-48db-b6a7-f9fdda42ef6e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:42.922027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:42.957501 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c75d7e-e5ff-4fad-8054-7b8ecac7a16d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:43.008724 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1039e9-0639-431d-9ea5-db327c72db92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:43.009445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1039e9-0639-431d-9ea5-db327c72db92 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:43.009597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1039e9-0639-431d-9ea5-db327c72db92 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:43.010365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1039e9-0639-431d-9ea5-db327c72db92 None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:43.013613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1039e9-0639-431d-9ea5-db327c72db92 None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:43.015073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1039e9-0639-431d-9ea5-db327c72db92 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:43.022888 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 210/409] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:42 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:43.023740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1039e9-0639-431d-9ea5-db327c72db92 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:43.093272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d652ae0-fd09-48db-b6a7-f9fdda42ef6e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:43.122637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:43.131366 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9a9d8-49cf-4fb9-982c-73817108f035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:43.132221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9a9d8-49cf-4fb9-982c-73817108f035 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:43.132653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9a9d8-49cf-4fb9-982c-73817108f035 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:43.135128 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9a9d8-49cf-4fb9-982c-73817108f035 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:43.135128 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9a9d8-49cf-4fb9-982c-73817108f035 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:43.135128 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9a9d8-49cf-4fb9-982c-73817108f035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:43.136112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9a9d8-49cf-4fb9-982c-73817108f035 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:43.148272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 200/410] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:43.149907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeb209-9f6d-40af-b0ad-5bb2de926902 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:43.150346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeb209-9f6d-40af-b0ad-5bb2de926902 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:43.150932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeb209-9f6d-40af-b0ad-5bb2de926902 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:43.152722 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 201/411] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:43.154199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eeb209-9f6d-40af-b0ad-5bb2de926902 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:43.178438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1039e9-0639-431d-9ea5-db327c72db92 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:43.214299 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f25aff-7cd6-4255-b784-74a289d439a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:43.215207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f25aff-7cd6-4255-b784-74a289d439a6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:43.215629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f25aff-7cd6-4255-b784-74a289d439a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:43.216080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f25aff-7cd6-4255-b784-74a289d439a6 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:43.217385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f25aff-7cd6-4255-b784-74a289d439a6 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:43.218948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f25aff-7cd6-4255-b784-74a289d439a6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:43.224912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 211/412] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:43 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:43.226399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e77705-e2d9-4363-9005-f1ee563aa7f0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:43.226822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e77705-e2d9-4363-9005-f1ee563aa7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:43.227256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e77705-e2d9-4363-9005-f1ee563aa7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:43.237940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:43.250535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-10e77705-e2d9-4363-9005-f1ee563aa7f0 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:43.251942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-10e77705-e2d9-4363-9005-f1ee563aa7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:43.294778 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eeb209-9f6d-40af-b0ad-5bb2de926902 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:43.333084 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 212/413] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:43.334581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e77705-e2d9-4363-9005-f1ee563aa7f0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:43.342177 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15342c6-22dc-4c7b-a847-067d4a99916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:43.342827 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15342c6-22dc-4c7b-a847-067d4a99916d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:43.343103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15342c6-22dc-4c7b-a847-067d4a99916d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:43.343385 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15342c6-22dc-4c7b-a847-067d4a99916d None admin] PATH_INFO: `/v3/groups/179572c9f41d4b819119233e892d2344` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:43.344590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15342c6-22dc-4c7b-a847-067d4a99916d None admin] RBAC: Authorizing `identity:get_group(group_id=179572c9f41d4b819119233e892d2344)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:43.346595 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15342c6-22dc-4c7b-a847-067d4a99916d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:43.349554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 202/414] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:43 2022] GET /identity/v3/groups/179572c9f41d4b819119233e892d2344 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:43.361042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15342c6-22dc-4c7b-a847-067d4a99916d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:43.423956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:43.447650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e77705-e2d9-4363-9005-f1ee563aa7f0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:43.470021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15342c6-22dc-4c7b-a847-067d4a99916d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:43.485564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61063a0f-40e5-455a-a6ca-e6722e0181b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:43.486615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61063a0f-40e5-455a-a6ca-e6722e0181b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:43.487149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61063a0f-40e5-455a-a6ca-e6722e0181b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:43.487670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61063a0f-40e5-455a-a6ca-e6722e0181b9 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389/groups/58bbc909690d4880b585e3eb7ae6efab/roles/e3c5a858a45c41db8ff3e883788d75bc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:43.490484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61063a0f-40e5-455a-a6ca-e6722e0181b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a913e1d666748509269614e0c708389, group_id=58bbc909690d4880b585e3eb7ae6efab, role_id=e3c5a858a45c41db8ff3e883788d75bc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:43.492339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61063a0f-40e5-455a-a6ca-e6722e0181b9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:43.495114 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:43.509167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be241b64-d209-453d-8b17-1df0731bc3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:43.509820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be241b64-d209-453d-8b17-1df0731bc3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:43.510105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be241b64-d209-453d-8b17-1df0731bc3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:43.510455 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be241b64-d209-453d-8b17-1df0731bc3a9 None admin] PATH_INFO: `/v3/projects/60367118cabe491a9e4ea2e0f6945aa3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:43.511621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be241b64-d209-453d-8b17-1df0731bc3a9 None admin] RBAC: Authorizing `identity:get_project(project_id=60367118cabe491a9e4ea2e0f6945aa3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:43.513182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be241b64-d209-453d-8b17-1df0731bc3a9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:43.515037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 203/415] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:43 2022] GET /identity/v3/projects/60367118cabe491a9e4ea2e0f6945aa3 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:43.515624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be241b64-d209-453d-8b17-1df0731bc3a9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:43.561333 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 213/416] 10.222.0.64 () {58 vars in 1442 bytes} [Thu Aug 4 07:23:43 2022] PUT /identity/v3/projects/6a913e1d666748509269614e0c708389/groups/58bbc909690d4880b585e3eb7ae6efab/roles/e3c5a858a45c41db8ff3e883788d75bc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:43.561869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61063a0f-40e5-455a-a6ca-e6722e0181b9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:43.672912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be241b64-d209-453d-8b17-1df0731bc3a9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:43.737487 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:43.751932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec37eb-1e06-4c83-af36-1dcf283ed7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:43.756084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec37eb-1e06-4c83-af36-1dcf283ed7df None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:43.756084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec37eb-1e06-4c83-af36-1dcf283ed7df None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:43.756084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec37eb-1e06-4c83-af36-1dcf283ed7df None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:43.756630 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61063a0f-40e5-455a-a6ca-e6722e0181b9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:43.764681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec37eb-1e06-4c83-af36-1dcf283ed7df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b0ec37eb-1e06-4c83-af36-1dcf283ed7df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:43.770059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:43.771204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:43.772046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 204/417] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:43.781062 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec37eb-1e06-4c83-af36-1dcf283ed7df None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:43.805583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcbd69-5ea3-4d20-ad84-9c474fcbe595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:43.806843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcbd69-5ea3-4d20-ad84-9c474fcbe595 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:43.807720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcbd69-5ea3-4d20-ad84-9c474fcbe595 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:43.808248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcbd69-5ea3-4d20-ad84-9c474fcbe595 None admin] PATH_INFO: `/v3/projects/60367118cabe491a9e4ea2e0f6945aa3/groups/179572c9f41d4b819119233e892d2344/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:43.811024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfcbd69-5ea3-4d20-ad84-9c474fcbe595 None admin] RBAC: Authorizing `identity:create_grant(project_id=60367118cabe491a9e4ea2e0f6945aa3, group_id=179572c9f41d4b819119233e892d2344, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:43.813303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfcbd69-5ea3-4d20-ad84-9c474fcbe595 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:43.877568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 214/418] 10.222.0.64 () {58 vars in 1442 bytes} [Thu Aug 4 07:23:43 2022] PUT /identity/v3/projects/60367118cabe491a9e4ea2e0f6945aa3/groups/179572c9f41d4b819119233e892d2344/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:23:43.914032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec37eb-1e06-4c83-af36-1dcf283ed7df None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:43.953377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dee5d8-7ba4-405f-bbd1-732fc8737151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:43.954425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dee5d8-7ba4-405f-bbd1-732fc8737151 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:43.954909 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dee5d8-7ba4-405f-bbd1-732fc8737151 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:43.955385 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dee5d8-7ba4-405f-bbd1-732fc8737151 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:43.956334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dee5d8-7ba4-405f-bbd1-732fc8737151 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:43.956846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dee5d8-7ba4-405f-bbd1-732fc8737151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:43.959576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dee5d8-7ba4-405f-bbd1-732fc8737151 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:43.970729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 205/419] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:43.976496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcbd69-5ea3-4d20-ad84-9c474fcbe595 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:44.083743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:44.133720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcbd69-5ea3-4d20-ad84-9c474fcbe595 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:44.172728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55615103-5fb8-4724-a224-733c7f1abdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:44.173353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55615103-5fb8-4724-a224-733c7f1abdca None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:44.173637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55615103-5fb8-4724-a224-733c7f1abdca None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:44.173912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55615103-5fb8-4724-a224-733c7f1abdca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:44.247566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55615103-5fb8-4724-a224-733c7f1abdca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:44.249064 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55615103-5fb8-4724-a224-733c7f1abdca None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:44.306185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:44.358419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 215/420] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:44.365412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dee5d8-7ba4-405f-bbd1-732fc8737151 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:44.436150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:44.491390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dee5d8-7ba4-405f-bbd1-732fc8737151 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:44.527755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db401f36-8d65-46d0-a593-afbdb750c1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:44.528779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db401f36-8d65-46d0-a593-afbdb750c1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:44.529302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db401f36-8d65-46d0-a593-afbdb750c1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:44.529779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db401f36-8d65-46d0-a593-afbdb750c1c1 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:44.531142 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db401f36-8d65-46d0-a593-afbdb750c1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:44.532797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db401f36-8d65-46d0-a593-afbdb750c1c1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:44.534662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 206/421] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:44 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:44.541351 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55615103-5fb8-4724-a224-733c7f1abdca None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:44.600973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e684c-83e5-47ce-b905-0c7b17ee3441 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:44.601594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e684c-83e5-47ce-b905-0c7b17ee3441 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:44.602103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e684c-83e5-47ce-b905-0c7b17ee3441 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:44.603650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 207/422] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:23:44.609037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d459817-d7b8-47bf-8379-525ea90120e5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:44.609256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d459817-d7b8-47bf-8379-525ea90120e5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:44.609539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d459817-d7b8-47bf-8379-525ea90120e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:44.651854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6d459817-d7b8-47bf-8379-525ea90120e5 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:44.653078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6d459817-d7b8-47bf-8379-525ea90120e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:44.679684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55615103-5fb8-4724-a224-733c7f1abdca None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:44.715254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93b590-7d0b-4d40-8583-ce9088b795a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:44.715894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93b590-7d0b-4d40-8583-ce9088b795a9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:44.716237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93b590-7d0b-4d40-8583-ce9088b795a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:44.716519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93b590-7d0b-4d40-8583-ce9088b795a9 None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:44.717741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93b590-7d0b-4d40-8583-ce9088b795a9 None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:44.719265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93b590-7d0b-4d40-8583-ce9088b795a9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:44.726434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 216/423] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:44 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:44.727635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93b590-7d0b-4d40-8583-ce9088b795a9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:44.759675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:44.796084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 208/424] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:44.830606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93b590-7d0b-4d40-8583-ce9088b795a9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:44.845968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d459817-d7b8-47bf-8379-525ea90120e5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:44.872649 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633150e-a60d-4d9c-a9f8-998df348eee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:44.873235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633150e-a60d-4d9c-a9f8-998df348eee7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:44.873525 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633150e-a60d-4d9c-a9f8-998df348eee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:44.873803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633150e-a60d-4d9c-a9f8-998df348eee7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:44.874602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633150e-a60d-4d9c-a9f8-998df348eee7 None admin] RBAC: Adding query filter params (group.id=58bbc909690d4880b585e3eb7ae6efab, role.id=525e499f44594268b4f362c62bd35e00, scope.project.id=ce53e46475f341a2af8bf9da4700fffe) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:44.874918 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633150e-a60d-4d9c-a9f8-998df348eee7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:44.876391 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633150e-a60d-4d9c-a9f8-998df348eee7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:44.896312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 217/425] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Aug 4 07:23:44 2022] GET /identity/v3/role_assignments?group.id=58bbc909690d4880b585e3eb7ae6efab&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=525e499f44594268b4f362c62bd35e00 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:44.936392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:45.006408 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d459817-d7b8-47bf-8379-525ea90120e5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:45.060137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388f26f-ca25-4ab8-b80f-3daaf2b3ffbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:45.060807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388f26f-ca25-4ab8-b80f-3daaf2b3ffbf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:45.061094 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388f26f-ca25-4ab8-b80f-3daaf2b3ffbf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:45.061374 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388f26f-ca25-4ab8-b80f-3daaf2b3ffbf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:45.073494 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388f26f-ca25-4ab8-b80f-3daaf2b3ffbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3388f26f-ca25-4ab8-b80f-3daaf2b3ffbf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:45.078558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:45.079421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:45.080256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 209/426] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:45.090536 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633150e-a60d-4d9c-a9f8-998df348eee7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:45.208370 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daf306-e2c7-465a-a57f-540cd1d7b2b8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:45.208737 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daf306-e2c7-465a-a57f-540cd1d7b2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:45.209018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daf306-e2c7-465a-a57f-540cd1d7b2b8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:45.210582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 210/427] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:23:45.220012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b3673-9ab9-4174-ab79-47ce19618642 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:45.220012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b3673-9ab9-4174-ab79-47ce19618642 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:45.220012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b3673-9ab9-4174-ab79-47ce19618642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:45.240168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cb5b3673-9ab9-4174-ab79-47ce19618642 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:45.242367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cb5b3673-9ab9-4174-ab79-47ce19618642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:45.248341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:45.250690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633150e-a60d-4d9c-a9f8-998df348eee7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:45.296403 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4fe0f-a119-453f-8697-ca10929b9791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:45.297036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4fe0f-a119-453f-8697-ca10929b9791 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:45.297320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4fe0f-a119-453f-8697-ca10929b9791 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:45.297598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4fe0f-a119-453f-8697-ca10929b9791 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:45.298369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4fe0f-a119-453f-8697-ca10929b9791 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:45.298624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4fe0f-a119-453f-8697-ca10929b9791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:45.300139 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4fe0f-a119-453f-8697-ca10929b9791 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:45.310642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 218/428] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:45.319858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4fe0f-a119-453f-8697-ca10929b9791 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:45.332150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 211/429] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:45.356430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b3673-9ab9-4174-ab79-47ce19618642 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:45.424500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:45.440380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4fe0f-a119-453f-8697-ca10929b9791 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:45.479928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b3673-9ab9-4174-ab79-47ce19618642 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:45.485930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d1567-04e7-4b03-920d-5fce714a9ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:45.486625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d1567-04e7-4b03-920d-5fce714a9ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:45.486980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d1567-04e7-4b03-920d-5fce714a9ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:45.487332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d1567-04e7-4b03-920d-5fce714a9ca2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:45.521653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:45.524196 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce34c2-5328-4124-ad8e-4891d86e9287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:45.525386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce34c2-5328-4124-ad8e-4891d86e9287 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:45.526010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce34c2-5328-4124-ad8e-4891d86e9287 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:45.526573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce34c2-5328-4124-ad8e-4891d86e9287 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:45.539804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce34c2-5328-4124-ad8e-4891d86e9287 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-bfce34c2-5328-4124-ad8e-4891d86e9287 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:45.549713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:45.550680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:45.552368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 212/430] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:45.560437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce34c2-5328-4124-ad8e-4891d86e9287 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:45.587295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d1567-04e7-4b03-920d-5fce714a9ca2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:45.588930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d1567-04e7-4b03-920d-5fce714a9ca2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:45.679714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:45.687221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce34c2-5328-4124-ad8e-4891d86e9287 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:45.709098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 219/431] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:45.714876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d1567-04e7-4b03-920d-5fce714a9ca2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:45.732539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cd04b-b9e5-4fc2-8f9b-6cfe8e193e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:45.733928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd04b-b9e5-4fc2-8f9b-6cfe8e193e05 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:45.734372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd04b-b9e5-4fc2-8f9b-6cfe8e193e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:45.734811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd04b-b9e5-4fc2-8f9b-6cfe8e193e05 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:45.736282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cd04b-b9e5-4fc2-8f9b-6cfe8e193e05 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:45.736760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cd04b-b9e5-4fc2-8f9b-6cfe8e193e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:45.739320 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cd04b-b9e5-4fc2-8f9b-6cfe8e193e05 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:45.754152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 213/432] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:23:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:45.762041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cd04b-b9e5-4fc2-8f9b-6cfe8e193e05 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:45.829183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:45.838237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d1567-04e7-4b03-920d-5fce714a9ca2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:45.876866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cd04b-b9e5-4fc2-8f9b-6cfe8e193e05 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:45.878013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab31716-e503-459b-89f2-ac398b8e18b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:45.880269 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab31716-e503-459b-89f2-ac398b8e18b4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:45.880563 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab31716-e503-459b-89f2-ac398b8e18b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:45.880847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab31716-e503-459b-89f2-ac398b8e18b4 None admin] PATH_INFO: `/v3/projects/ce53e46475f341a2af8bf9da4700fffe` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:45.884753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab31716-e503-459b-89f2-ac398b8e18b4 None admin] RBAC: Authorizing `identity:get_project(project_id=ce53e46475f341a2af8bf9da4700fffe)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:45.887022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab31716-e503-459b-89f2-ac398b8e18b4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:45.890173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 220/433] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:45 2022] GET /identity/v3/projects/ce53e46475f341a2af8bf9da4700fffe => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:45.895973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab31716-e503-459b-89f2-ac398b8e18b4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:45.919457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038c2d9-3624-48a1-8008-6e02b461e568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:45.920380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038c2d9-3624-48a1-8008-6e02b461e568 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:45.920801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038c2d9-3624-48a1-8008-6e02b461e568 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:45.921239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038c2d9-3624-48a1-8008-6e02b461e568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:45.960602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:46.016320 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038c2d9-3624-48a1-8008-6e02b461e568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:46.017690 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038c2d9-3624-48a1-8008-6e02b461e568 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:46.021471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab31716-e503-459b-89f2-ac398b8e18b4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:46.038680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:46.057150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd520b-ddd7-4a9a-85a3-3297bf6f2532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:46.057789 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd520b-ddd7-4a9a-85a3-3297bf6f2532 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.058492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd520b-ddd7-4a9a-85a3-3297bf6f2532 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.061052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd520b-ddd7-4a9a-85a3-3297bf6f2532 None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.061052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd520b-ddd7-4a9a-85a3-3297bf6f2532 None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:46.062223 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd520b-ddd7-4a9a-85a3-3297bf6f2532 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:46.068945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 221/434] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:45 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:46.068945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1c592-98aa-406b-841d-ad579695008b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.068945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1c592-98aa-406b-841d-ad579695008b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.068945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1c592-98aa-406b-841d-ad579695008b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.068945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 222/435] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:46.071736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1c592-98aa-406b-841d-ad579695008b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:46.138866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 214/436] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 07:23:46.140186 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773519d8-c96e-475c-af63-1cad038b6739 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.140430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773519d8-c96e-475c-af63-1cad038b6739 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.140709 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773519d8-c96e-475c-af63-1cad038b6739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.159296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-773519d8-c96e-475c-af63-1cad038b6739 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:46.160543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-773519d8-c96e-475c-af63-1cad038b6739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:46.184608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1c592-98aa-406b-841d-ad579695008b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:46.217115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:46.225412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d52c3-78e3-45f4-9ddc-b90c9ff1f8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:46.227333 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d52c3-78e3-45f4-9ddc-b90c9ff1f8ac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.227333 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d52c3-78e3-45f4-9ddc-b90c9ff1f8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.227333 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d52c3-78e3-45f4-9ddc-b90c9ff1f8ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.227732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d52c3-78e3-45f4-9ddc-b90c9ff1f8ac None admin] RBAC: Adding query filter params (group.id=58bbc909690d4880b585e3eb7ae6efab, role.id=e3c5a858a45c41db8ff3e883788d75bc, scope.project.id=ce53e46475f341a2af8bf9da4700fffe) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:46.227732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d52c3-78e3-45f4-9ddc-b90c9ff1f8ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:46.229164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d52c3-78e3-45f4-9ddc-b90c9ff1f8ac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:46.232377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 215/437] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:46.232977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773519d8-c96e-475c-af63-1cad038b6739 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:46.244093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 223/438] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Aug 4 07:23:46 2022] GET /identity/v3/role_assignments?group.id=58bbc909690d4880b585e3eb7ae6efab&scope.project.id=ce53e46475f341a2af8bf9da4700fffe&role.id=e3c5a858a45c41db8ff3e883788d75bc => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:46.273573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d52c3-78e3-45f4-9ddc-b90c9ff1f8ac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:46.346027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:46.352950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773519d8-c96e-475c-af63-1cad038b6739 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:46.395718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a319c-51c3-463f-8060-a987e2613996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:46.396661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a319c-51c3-463f-8060-a987e2613996 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.397112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a319c-51c3-463f-8060-a987e2613996 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.397540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a319c-51c3-463f-8060-a987e2613996 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.398904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a319c-51c3-463f-8060-a987e2613996 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:46.400586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a319c-51c3-463f-8060-a987e2613996 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:46.402539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 216/439] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:46 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:46.403837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dc17e-382c-4b44-b9a6-d1e5d68d7493 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.404293 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dc17e-382c-4b44-b9a6-d1e5d68d7493 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.404719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dc17e-382c-4b44-b9a6-d1e5d68d7493 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.406308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 217/440] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:46.408175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35dc17e-382c-4b44-b9a6-d1e5d68d7493 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:46.412881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d52c3-78e3-45f4-9ddc-b90c9ff1f8ac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:46.455967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459bf1e-5dc6-48d4-98d8-e6c7c7e64131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:46.456655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459bf1e-5dc6-48d4-98d8-e6c7c7e64131 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.456942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459bf1e-5dc6-48d4-98d8-e6c7c7e64131 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.457220 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459bf1e-5dc6-48d4-98d8-e6c7c7e64131 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.474186 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459bf1e-5dc6-48d4-98d8-e6c7c7e64131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:46.478972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2459bf1e-5dc6-48d4-98d8-e6c7c7e64131 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:46.480693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:46.481590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:46.482398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 224/441] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:46.483910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23c018-1029-4e7a-adf6-74a69f59814d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.483910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23c018-1029-4e7a-adf6-74a69f59814d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.483910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23c018-1029-4e7a-adf6-74a69f59814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.507872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0a23c018-1029-4e7a-adf6-74a69f59814d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:46.509113 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0a23c018-1029-4e7a-adf6-74a69f59814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:46.534547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35dc17e-382c-4b44-b9a6-d1e5d68d7493 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:46.574231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f0367-21a5-4212-8df6-0d1fd44ea685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:46.574878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0367-21a5-4212-8df6-0d1fd44ea685 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.575154 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0367-21a5-4212-8df6-0d1fd44ea685 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.575425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0367-21a5-4212-8df6-0d1fd44ea685 None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.576594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f0367-21a5-4212-8df6-0d1fd44ea685 None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:46.578022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f0367-21a5-4212-8df6-0d1fd44ea685 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:46.579604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 218/442] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:46 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:46.580383 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f0367-21a5-4212-8df6-0d1fd44ea685 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:46.594496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 225/443] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:46.595331 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23c018-1029-4e7a-adf6-74a69f59814d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:46.646135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:46.694504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f0367-21a5-4212-8df6-0d1fd44ea685 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:46.704389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23c018-1029-4e7a-adf6-74a69f59814d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:46.717434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:46.728203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7ada2-55a7-4a0f-9970-be71677a2a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:46.731112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7ada2-55a7-4a0f-9970-be71677a2a4d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.731112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7ada2-55a7-4a0f-9970-be71677a2a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.731112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7ada2-55a7-4a0f-9970-be71677a2a4d None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.731112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7ada2-55a7-4a0f-9970-be71677a2a4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:46.731112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7ada2-55a7-4a0f-9970-be71677a2a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:46.731863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7ada2-55a7-4a0f-9970-be71677a2a4d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:46.738435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16754713-fa29-45d1-a05b-fad2e5985640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:46.739948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16754713-fa29-45d1-a05b-fad2e5985640 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.740262 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16754713-fa29-45d1-a05b-fad2e5985640 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.740543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16754713-fa29-45d1-a05b-fad2e5985640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.741069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 219/444] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:46.741340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16754713-fa29-45d1-a05b-fad2e5985640 None admin] RBAC: Adding query filter params (group.id=58bbc909690d4880b585e3eb7ae6efab, role.id=525e499f44594268b4f362c62bd35e00, scope.project.id=6a913e1d666748509269614e0c708389) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:46.741655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16754713-fa29-45d1-a05b-fad2e5985640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:46.741852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7ada2-55a7-4a0f-9970-be71677a2a4d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:46.743170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16754713-fa29-45d1-a05b-fad2e5985640 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:46.759587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 226/445] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Aug 4 07:23:46 2022] GET /identity/v3/role_assignments?group.id=58bbc909690d4880b585e3eb7ae6efab&scope.project.id=6a913e1d666748509269614e0c708389&role.id=525e499f44594268b4f362c62bd35e00 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:46.759587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16754713-fa29-45d1-a05b-fad2e5985640 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:46.859805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7ada2-55a7-4a0f-9970-be71677a2a4d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:46.862801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16754713-fa29-45d1-a05b-fad2e5985640 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:46.865644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:46.897020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fc4e-ea09-4ee5-ac3c-44e77be12aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:46.897691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fc4e-ea09-4ee5-ac3c-44e77be12aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.897946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fc4e-ea09-4ee5-ac3c-44e77be12aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.898928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fc4e-ea09-4ee5-ac3c-44e77be12aa0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.903071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0186b-4a1b-4cba-9d43-6228ce8bdce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:46.903810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0186b-4a1b-4cba-9d43-6228ce8bdce3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:46.904579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0186b-4a1b-4cba-9d43-6228ce8bdce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:46.904579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0186b-4a1b-4cba-9d43-6228ce8bdce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:46.911647 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8fc4e-ea09-4ee5-ac3c-44e77be12aa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-93e8fc4e-ea09-4ee5-ac3c-44e77be12aa0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:46.917665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:46.918678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:23:46.919491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:46.923781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 220/446] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:23:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:46.924988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fc4e-ea09-4ee5-ac3c-44e77be12aa0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:46.983105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:46.993632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0186b-4a1b-4cba-9d43-6228ce8bdce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:46.995231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0186b-4a1b-4cba-9d43-6228ce8bdce3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:47.050508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fc4e-ea09-4ee5-ac3c-44e77be12aa0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:47.097488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc26c8b-392e-4906-8637-a8a51e33d4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:47.098737 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc26c8b-392e-4906-8637-a8a51e33d4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:47.099021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc26c8b-392e-4906-8637-a8a51e33d4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:47.099298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc26c8b-392e-4906-8637-a8a51e33d4b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:47.100048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc26c8b-392e-4906-8637-a8a51e33d4b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:47.100356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc26c8b-392e-4906-8637-a8a51e33d4b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:47.101740 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc26c8b-392e-4906-8637-a8a51e33d4b7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:47.111045 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 221/447] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:47.112706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 227/448] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:47.116717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc26c8b-392e-4906-8637-a8a51e33d4b7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:47.118475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0186b-4a1b-4cba-9d43-6228ce8bdce3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:47.120523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:47.215141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc26c8b-392e-4906-8637-a8a51e33d4b7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:47.217328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0186b-4a1b-4cba-9d43-6228ce8bdce3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:47.224593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:47.250291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05240f-a572-4eb8-8528-45e041392296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:47.250461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bb996-3b1b-4702-8ede-c29c4610cab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:47.250911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05240f-a572-4eb8-8528-45e041392296 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:47.251047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bb996-3b1b-4702-8ede-c29c4610cab8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:47.251196 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05240f-a572-4eb8-8528-45e041392296 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:47.251332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bb996-3b1b-4702-8ede-c29c4610cab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:47.251475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05240f-a572-4eb8-8528-45e041392296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:47.251611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bb996-3b1b-4702-8ede-c29c4610cab8 None admin] PATH_INFO: `/v3/projects/6a913e1d666748509269614e0c708389` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:47.252785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bb996-3b1b-4702-8ede-c29c4610cab8 None admin] RBAC: Authorizing `identity:get_project(project_id=6a913e1d666748509269614e0c708389)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:47.254516 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bb996-3b1b-4702-8ede-c29c4610cab8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:47.256192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 228/449] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:47 2022] GET /identity/v3/projects/6a913e1d666748509269614e0c708389 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:47.261586 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bb996-3b1b-4702-8ede-c29c4610cab8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:47.320783 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05240f-a572-4eb8-8528-45e041392296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:47.322239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05240f-a572-4eb8-8528-45e041392296 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:47.330124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:47.361334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bb996-3b1b-4702-8ede-c29c4610cab8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:47.394080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daa6b3-1bf1-4515-8d92-0c4de3c939aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:47.394732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daa6b3-1bf1-4515-8d92-0c4de3c939aa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:47.395043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daa6b3-1bf1-4515-8d92-0c4de3c939aa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:47.395319 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daa6b3-1bf1-4515-8d92-0c4de3c939aa None admin] PATH_INFO: `/v3/groups/58bbc909690d4880b585e3eb7ae6efab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:47.396490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daa6b3-1bf1-4515-8d92-0c4de3c939aa None admin] RBAC: Authorizing `identity:get_group(group_id=58bbc909690d4880b585e3eb7ae6efab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:47.397926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daa6b3-1bf1-4515-8d92-0c4de3c939aa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:47.399497 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 229/450] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:47 2022] GET /identity/v3/groups/58bbc909690d4880b585e3eb7ae6efab => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:47.405171 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daa6b3-1bf1-4515-8d92-0c4de3c939aa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:47.420840 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 222/451] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:47.426763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05240f-a572-4eb8-8528-45e041392296 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:47.435799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:47.501984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daa6b3-1bf1-4515-8d92-0c4de3c939aa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:47.524427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05240f-a572-4eb8-8528-45e041392296 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:47.534828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f2e31-c0e4-433f-9980-a1beeb733b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:47.535934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f2e31-c0e4-433f-9980-a1beeb733b93 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:47.536246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f2e31-c0e4-433f-9980-a1beeb733b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:47.536519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f2e31-c0e4-433f-9980-a1beeb733b93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:47.537304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f2e31-c0e4-433f-9980-a1beeb733b93 None admin] RBAC: Adding query filter params (group.id=58bbc909690d4880b585e3eb7ae6efab, role.id=e3c5a858a45c41db8ff3e883788d75bc, scope.project.id=6a913e1d666748509269614e0c708389) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:47.537615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f2e31-c0e4-433f-9980-a1beeb733b93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:47.539120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f2e31-c0e4-433f-9980-a1beeb733b93 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:47.544840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 230/452] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Aug 4 07:23:47 2022] GET /identity/v3/role_assignments?group.id=58bbc909690d4880b585e3eb7ae6efab&scope.project.id=6a913e1d666748509269614e0c708389&role.id=e3c5a858a45c41db8ff3e883788d75bc => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:47.558668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa7686-ca40-4c7a-88dc-26d82f51e83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:47.559367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa7686-ca40-4c7a-88dc-26d82f51e83e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:47.559659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa7686-ca40-4c7a-88dc-26d82f51e83e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:47.559937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa7686-ca40-4c7a-88dc-26d82f51e83e None admin] PATH_INFO: `/v3/projects/60367118cabe491a9e4ea2e0f6945aa3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:47.561233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa7686-ca40-4c7a-88dc-26d82f51e83e None admin] RBAC: Authorizing `identity:get_project(project_id=60367118cabe491a9e4ea2e0f6945aa3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:47.562739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa7686-ca40-4c7a-88dc-26d82f51e83e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:47.564522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 223/453] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:23:47 2022] GET /identity/v3/projects/60367118cabe491a9e4ea2e0f6945aa3 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:47.570599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f2e31-c0e4-433f-9980-a1beeb733b93 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:47.573591 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:47.676165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f2e31-c0e4-433f-9980-a1beeb733b93 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:47.711055 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b54e0-06e8-4c0a-bf2b-3e8cbeea2374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:47.711694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b54e0-06e8-4c0a-bf2b-3e8cbeea2374 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:47.711977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b54e0-06e8-4c0a-bf2b-3e8cbeea2374 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:47.712270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b54e0-06e8-4c0a-bf2b-3e8cbeea2374 None admin] PATH_INFO: `/v3/groups/179572c9f41d4b819119233e892d2344` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:47.713410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b54e0-06e8-4c0a-bf2b-3e8cbeea2374 None admin] RBAC: Authorizing `identity:get_group(group_id=179572c9f41d4b819119233e892d2344)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:47.714847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b54e0-06e8-4c0a-bf2b-3e8cbeea2374 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:47.716500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 231/454] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Aug 4 07:23:47 2022] GET /identity/v3/groups/179572c9f41d4b819119233e892d2344 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:47.722492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa7686-ca40-4c7a-88dc-26d82f51e83e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:47.803864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:47.826973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa7686-ca40-4c7a-88dc-26d82f51e83e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:47.861463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2513244-b826-4e4f-bb6a-6def5ac3be75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:47.862108 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2513244-b826-4e4f-bb6a-6def5ac3be75 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:47.862396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2513244-b826-4e4f-bb6a-6def5ac3be75 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:47.862677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2513244-b826-4e4f-bb6a-6def5ac3be75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:47.864065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2513244-b826-4e4f-bb6a-6def5ac3be75 None admin] RBAC: Adding query filter params (group.id=179572c9f41d4b819119233e892d2344, role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=60367118cabe491a9e4ea2e0f6945aa3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:47.864467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2513244-b826-4e4f-bb6a-6def5ac3be75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:47.865903 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2513244-b826-4e4f-bb6a-6def5ac3be75 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:47.871777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 224/455] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Aug 4 07:23:47 2022] GET /identity/v3/role_assignments?group.id=179572c9f41d4b819119233e892d2344&scope.project.id=60367118cabe491a9e4ea2e0f6945aa3&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:50.237391 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8bc40-deb9-4218-abba-44357b348400 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:50.237919 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8bc40-deb9-4218-abba-44357b348400 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:50.237919 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8bc40-deb9-4218-abba-44357b348400 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:50.239815 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 232/456] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:50.245531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a8e4-842d-495f-a551-83c1116f8376 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:50.246482 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a8e4-842d-495f-a551-83c1116f8376 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:50.246912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a8e4-842d-495f-a551-83c1116f8376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:50.276811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-77e4a8e4-842d-495f-a551-83c1116f8376 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:50.278292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-77e4a8e4-842d-495f-a551-83c1116f8376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:50.370143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 225/457] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:50.411581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8bc40-deb9-4218-abba-44357b348400 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:50.509816 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:50.535826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb174d7-ac65-4576-bc53-012706477304 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:50.536480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb174d7-ac65-4576-bc53-012706477304 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:50.537000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb174d7-ac65-4576-bc53-012706477304 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:50.538810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 226/458] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:50.543686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dbabc-eb05-44a3-8220-8ce65f2e1471 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:50.544236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dbabc-eb05-44a3-8220-8ce65f2e1471 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:50.544988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dbabc-eb05-44a3-8220-8ce65f2e1471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:50.553330 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8bc40-deb9-4218-abba-44357b348400 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:50.571361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-666dbabc-eb05-44a3-8220-8ce65f2e1471 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:50.572588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-666dbabc-eb05-44a3-8220-8ce65f2e1471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:50.612681 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620c157-8b75-48ab-b22c-854f0840c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:50.613961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620c157-8b75-48ab-b22c-854f0840c4ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:50.614266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620c157-8b75-48ab-b22c-854f0840c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:50.614541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620c157-8b75-48ab-b22c-854f0840c4ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:50.631406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620c157-8b75-48ab-b22c-854f0840c4ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:50.632911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620c157-8b75-48ab-b22c-854f0840c4ce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7620c157-8b75-48ab-b22c-854f0840c4ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:50.658538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:50.659439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:50.660859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:50.660859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:50.660859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:50.660859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:50.660859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 233/459] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:50.667674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620c157-8b75-48ab-b22c-854f0840c4ce None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:50.697774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:50.708670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 227/460] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:50.732783 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dbabc-eb05-44a3-8220-8ce65f2e1471 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:50.837072 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620c157-8b75-48ab-b22c-854f0840c4ce None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:50.866159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:50.870491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dbabc-eb05-44a3-8220-8ce65f2e1471 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:50.880083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4ec9e-ac9b-4a81-9069-5d06a3174069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:50.881033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4ec9e-ac9b-4a81-9069-5d06a3174069 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:50.881902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4ec9e-ac9b-4a81-9069-5d06a3174069 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:50.882632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4ec9e-ac9b-4a81-9069-5d06a3174069 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:50.884189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4ec9e-ac9b-4a81-9069-5d06a3174069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:50.884803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4ec9e-ac9b-4a81-9069-5d06a3174069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:50.886204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4ec9e-ac9b-4a81-9069-5d06a3174069 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:50.904523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 234/461] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:50.904523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad20f86-448c-483a-b740-c3fa9f08db01 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:50.904523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad20f86-448c-483a-b740-c3fa9f08db01 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:50.904523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad20f86-448c-483a-b740-c3fa9f08db01 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:50.904523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 235/462] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:50.904523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e415b-8b19-4c8b-9363-f6e61df93e93 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:50.904973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e415b-8b19-4c8b-9363-f6e61df93e93 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:50.904973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e415b-8b19-4c8b-9363-f6e61df93e93 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:50.906417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 236/463] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:50.907093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e415b-8b19-4c8b-9363-f6e61df93e93 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:50.919268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20703afd-23d4-4154-85d9-ef043cbc555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:50.919911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703afd-23d4-4154-85d9-ef043cbc555f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:50.920232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703afd-23d4-4154-85d9-ef043cbc555f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:50.920507 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703afd-23d4-4154-85d9-ef043cbc555f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:50.932148 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20703afd-23d4-4154-85d9-ef043cbc555f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:50.933624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20703afd-23d4-4154-85d9-ef043cbc555f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-20703afd-23d4-4154-85d9-ef043cbc555f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:50.950609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:50.951481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:50.956975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:50.956975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:50.956975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:50.956975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:50.956975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 228/464] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:50.956975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f3eb-8356-48f5-ab99-bd675bad10ea None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:50.956975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f3eb-8356-48f5-ab99-bd675bad10ea None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:50.956975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f3eb-8356-48f5-ab99-bd675bad10ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:50.972861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a4c4f3eb-8356-48f5-ab99-bd675bad10ea None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:50.974306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a4c4f3eb-8356-48f5-ab99-bd675bad10ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:51.023520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e415b-8b19-4c8b-9363-f6e61df93e93 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:51.027619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:51.049051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 229/465] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:51.050314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f0524-f8b7-4487-841a-530979aa9581 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:51.050598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f0524-f8b7-4487-841a-530979aa9581 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:51.051398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f0524-f8b7-4487-841a-530979aa9581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:51.073342 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c06f0524-f8b7-4487-841a-530979aa9581 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:51.075249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c06f0524-f8b7-4487-841a-530979aa9581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:51.075724 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fc470-f5da-4524-a51a-e05c913558fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:51.076449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fc470-f5da-4524-a51a-e05c913558fe None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:51.076802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fc470-f5da-4524-a51a-e05c913558fe None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:51.077151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fc470-f5da-4524-a51a-e05c913558fe None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:51.077963 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fc470-f5da-4524-a51a-e05c913558fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:51.079837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fc470-f5da-4524-a51a-e05c913558fe None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:51.142961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 230/466] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:51.144078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f0524-f8b7-4487-841a-530979aa9581 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:51.173574 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 237/467] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:23:50 2022] POST /identity/v3/users => generated 246 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:23:51.174249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fc470-f5da-4524-a51a-e05c913558fe None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:51.241099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:51.269025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f0524-f8b7-4487-841a-530979aa9581 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:51.289119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:51.313359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fc470-f5da-4524-a51a-e05c913558fe None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:51.319198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d7d71-7caa-4610-9eae-0190a51b3853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:51.320225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d7d71-7caa-4610-9eae-0190a51b3853 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:51.320708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d7d71-7caa-4610-9eae-0190a51b3853 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:51.321217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d7d71-7caa-4610-9eae-0190a51b3853 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:51.322182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d7d71-7caa-4610-9eae-0190a51b3853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:51.323158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d7d71-7caa-4610-9eae-0190a51b3853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:51.325139 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d7d71-7caa-4610-9eae-0190a51b3853 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:51.341033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 231/468] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:51.342991 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d7d71-7caa-4610-9eae-0190a51b3853 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:51.354345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d2502-08d9-4819-a675-e56451fc335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:51.355200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d2502-08d9-4819-a675-e56451fc335d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:51.356100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d2502-08d9-4819-a675-e56451fc335d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:51.356100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d2502-08d9-4819-a675-e56451fc335d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:51.367555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d2502-08d9-4819-a675-e56451fc335d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:51.369491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d2502-08d9-4819-a675-e56451fc335d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bc4d2502-08d9-4819-a675-e56451fc335d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:51.386909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:51.387794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:51.388716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:51.388716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:51.388716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:51.388716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:51.388716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 238/469] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:51.388996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d2502-08d9-4819-a675-e56451fc335d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:51.472768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d7d71-7caa-4610-9eae-0190a51b3853 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:51.504507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:51.514960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5da65-3824-4704-8d4f-3bcaa5a23158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:51.515884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d2502-08d9-4819-a675-e56451fc335d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:51.516090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5da65-3824-4704-8d4f-3bcaa5a23158 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:51.516524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5da65-3824-4704-8d4f-3bcaa5a23158 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:51.516980 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5da65-3824-4704-8d4f-3bcaa5a23158 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:51.530684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5da65-3824-4704-8d4f-3bcaa5a23158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:51.532255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5da65-3824-4704-8d4f-3bcaa5a23158 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5ac5da65-3824-4704-8d4f-3bcaa5a23158 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:51.547893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:51.548807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:51.550273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:51.550273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:51.550273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:51.550273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:51.550273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 232/470] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:51.550273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5da65-3824-4704-8d4f-3bcaa5a23158 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:51.563437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bbac5-034c-49ba-80ee-1041d1196346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:51.564756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bbac5-034c-49ba-80ee-1041d1196346 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:51.565766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bbac5-034c-49ba-80ee-1041d1196346 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:51.566049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bbac5-034c-49ba-80ee-1041d1196346 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:51.566770 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bbac5-034c-49ba-80ee-1041d1196346 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:51.568373 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bbac5-034c-49ba-80ee-1041d1196346 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:51.584902 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:51.664042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 239/471] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:23:51 2022] POST /identity/v3/users => generated 248 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:23:51.664042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bbac5-034c-49ba-80ee-1041d1196346 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:51.675116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5da65-3824-4704-8d4f-3bcaa5a23158 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:51.751422 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfa9e1-a9b5-4e3c-b9e9-e5d8f392fd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:51.752009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfa9e1-a9b5-4e3c-b9e9-e5d8f392fd3c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:51.752299 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfa9e1-a9b5-4e3c-b9e9-e5d8f392fd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:51.752576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfa9e1-a9b5-4e3c-b9e9-e5d8f392fd3c None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:51.753295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfa9e1-a9b5-4e3c-b9e9-e5d8f392fd3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:51.753606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfa9e1-a9b5-4e3c-b9e9-e5d8f392fd3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:51.755216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfa9e1-a9b5-4e3c-b9e9-e5d8f392fd3c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:51.779077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 233/472] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:51.784863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfa9e1-a9b5-4e3c-b9e9-e5d8f392fd3c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:51.830259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bbac5-034c-49ba-80ee-1041d1196346 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:51.869347 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a41d3-884c-4cc8-8d92-96df08057d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:51.869347 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a41d3-884c-4cc8-8d92-96df08057d5e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:51.869347 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a41d3-884c-4cc8-8d92-96df08057d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:51.869715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a41d3-884c-4cc8-8d92-96df08057d5e None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:51.870248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a41d3-884c-4cc8-8d92-96df08057d5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:51.870559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a41d3-884c-4cc8-8d92-96df08057d5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:51.871960 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a41d3-884c-4cc8-8d92-96df08057d5e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:51.886092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 240/473] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:51.893045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a41d3-884c-4cc8-8d92-96df08057d5e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:51.935641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:51.950432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:51.975084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfa9e1-a9b5-4e3c-b9e9-e5d8f392fd3c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:52.024483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943542ff-7aa6-4ad5-837c-b19dc0df10e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:52.025029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943542ff-7aa6-4ad5-837c-b19dc0df10e2 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:52.025309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943542ff-7aa6-4ad5-837c-b19dc0df10e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:52.025576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943542ff-7aa6-4ad5-837c-b19dc0df10e2 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:52.026384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943542ff-7aa6-4ad5-837c-b19dc0df10e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:52.027743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943542ff-7aa6-4ad5-837c-b19dc0df10e2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:52.031380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a41d3-884c-4cc8-8d92-96df08057d5e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:52.070163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5e49a-cbd9-43c0-8fa8-ba07782067a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:52.070776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5e49a-cbd9-43c0-8fa8-ba07782067a1 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:52.071058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5e49a-cbd9-43c0-8fa8-ba07782067a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:52.071335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5e49a-cbd9-43c0-8fa8-ba07782067a1 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:52.072092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5e49a-cbd9-43c0-8fa8-ba07782067a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:52.073536 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5e49a-cbd9-43c0-8fa8-ba07782067a1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:52.139845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 234/474] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:23:51 2022] POST /identity/v3/users => generated 248 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:23:52.141480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934ad03-ce90-4d79-ac68-2abf9b054463 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:52.142126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934ad03-ce90-4d79-ac68-2abf9b054463 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:52.142614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934ad03-ce90-4d79-ac68-2abf9b054463 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:52.144474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 235/475] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:52.150610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ea736-b98d-44bc-9cac-db13acee6ce3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:52.151239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ea736-b98d-44bc-9cac-db13acee6ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:52.151717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ea736-b98d-44bc-9cac-db13acee6ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:52.174151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a57ea736-b98d-44bc-9cac-db13acee6ce3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:52.176733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a57ea736-b98d-44bc-9cac-db13acee6ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:52.206102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 241/476] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:23:51 2022] POST /identity/v3/users => generated 249 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:23:52.265593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 236/477] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:52.291236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5e49a-cbd9-43c0-8fa8-ba07782067a1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:52.380693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:52.435413 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5e49a-cbd9-43c0-8fa8-ba07782067a1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:52.474397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5be138-6b25-48c7-a210-bdcba04af40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:52.475027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5be138-6b25-48c7-a210-bdcba04af40e None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:52.475310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5be138-6b25-48c7-a210-bdcba04af40e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:52.475584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5be138-6b25-48c7-a210-bdcba04af40e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:52.476654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5be138-6b25-48c7-a210-bdcba04af40e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:52.477741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5be138-6b25-48c7-a210-bdcba04af40e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:52.534602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.sql.core [None req-db5be138-6b25-48c7-a210-bdcba04af40e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 07:23:52.534602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 07:23:52.534602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [parameters: {'id': 'a5f20f2d307b4f4ba08480f9613132ce', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 07:23:52.534602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80297) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-db5be138-6b25-48c7-a210-bdcba04af40e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Aug 04 07:23:52.551695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return ref.to_dict() Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 07:23:52.552692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application yield resource Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise self.value Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application session.commit() Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 07:23:52.553626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.flush() Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application compat.raise_( Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 07:23:52.554491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 07:23:52.555380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 07:23:52.557522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [parameters: {'id': 'a5f20f2d307b4f4ba08480f9613132ce', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:52.559564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 07:23:52.562471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:52.564724 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 242/478] 10.222.0.64 () {60 vars in 1131 bytes} [Thu Aug 4 07:23:52 2022] POST /identity/v3/roles => generated 144 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:52.568164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ea736-b98d-44bc-9cac-db13acee6ce3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:52.641555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:52.673057 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ea736-b98d-44bc-9cac-db13acee6ce3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:52.715460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f8e65-5bf5-4437-9a7f-19ebc9ff05d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:52.716150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f8e65-5bf5-4437-9a7f-19ebc9ff05d3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:52.716437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f8e65-5bf5-4437-9a7f-19ebc9ff05d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:52.716715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f8e65-5bf5-4437-9a7f-19ebc9ff05d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:52.731413 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f8e65-5bf5-4437-9a7f-19ebc9ff05d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ae9f8e65-5bf5-4437-9a7f-19ebc9ff05d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:52.736610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:52.739324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 07:23:52.741410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:52.748241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 237/479] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:23:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:52.749252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5be138-6b25-48c7-a210-bdcba04af40e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:52.887631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5be138-6b25-48c7-a210-bdcba04af40e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:52.919243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:52.927426 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ab1e6-6665-46c1-a03b-c8b842b1e7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:52.928076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ab1e6-6665-46c1-a03b-c8b842b1e7cd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:52.928360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ab1e6-6665-46c1-a03b-c8b842b1e7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:52.928639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ab1e6-6665-46c1-a03b-c8b842b1e7cd None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:52.929364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ab1e6-6665-46c1-a03b-c8b842b1e7cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:52.929699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ab1e6-6665-46c1-a03b-c8b842b1e7cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:52.931051 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ab1e6-6665-46c1-a03b-c8b842b1e7cd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:52.940152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 243/480] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:53.491757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf446-9020-48cd-ad8a-0e1935d051b4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:53.492206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf446-9020-48cd-ad8a-0e1935d051b4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:53.492386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf446-9020-48cd-ad8a-0e1935d051b4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:53.493981 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 238/481] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:53.499383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caee28-fd3d-434a-92c4-597495036884 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:53.499820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caee28-fd3d-434a-92c4-597495036884 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:53.500004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caee28-fd3d-434a-92c4-597495036884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:53.521012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e9caee28-fd3d-434a-92c4-597495036884 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:53.522213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e9caee28-fd3d-434a-92c4-597495036884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:53.610807 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 244/482] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:23:53.648185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bf446-9020-48cd-ad8a-0e1935d051b4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:53.785935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bf446-9020-48cd-ad8a-0e1935d051b4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:53.815721 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:53.825652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cf951-193a-4c1c-9a26-c6738af33e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:53.826298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf951-193a-4c1c-9a26-c6738af33e71 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:53.826583 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf951-193a-4c1c-9a26-c6738af33e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:53.836504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf951-193a-4c1c-9a26-c6738af33e71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:53.842696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cf951-193a-4c1c-9a26-c6738af33e71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8d7cf951-193a-4c1c-9a26-c6738af33e71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:53.847657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:53.848590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:23:53.849377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:53.853656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 239/483] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:23:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:53.854913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caee28-fd3d-434a-92c4-597495036884 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:54.019061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caee28-fd3d-434a-92c4-597495036884 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:54.043062 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:54.065340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860f0ff-0d55-4baa-a901-22852df26f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:54.065957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860f0ff-0d55-4baa-a901-22852df26f01 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:54.066302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860f0ff-0d55-4baa-a901-22852df26f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:54.067348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860f0ff-0d55-4baa-a901-22852df26f01 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:54.068110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860f0ff-0d55-4baa-a901-22852df26f01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:54.068421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860f0ff-0d55-4baa-a901-22852df26f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:54.069882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860f0ff-0d55-4baa-a901-22852df26f01 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:54.083898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 245/484] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:23:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:54.089805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cf951-193a-4c1c-9a26-c6738af33e71 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:54.220908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cf951-193a-4c1c-9a26-c6738af33e71 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:54.261372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009dabf-d895-4cda-9c18-94430292cb72 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:54.261721 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009dabf-d895-4cda-9c18-94430292cb72 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:54.261936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009dabf-d895-4cda-9c18-94430292cb72 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:54.264190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4030119-6037-41c6-a17f-bc5e3cd43e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:54.265068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4030119-6037-41c6-a17f-bc5e3cd43e1b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:54.265543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4030119-6037-41c6-a17f-bc5e3cd43e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:54.266008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4030119-6037-41c6-a17f-bc5e3cd43e1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:54.269082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 246/485] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:54.279731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e516302-8840-4357-aecf-e619fdd49d3b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:54.280106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e516302-8840-4357-aecf-e619fdd49d3b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:54.280387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e516302-8840-4357-aecf-e619fdd49d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:54.282320 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4030119-6037-41c6-a17f-bc5e3cd43e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:54.284022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4030119-6037-41c6-a17f-bc5e3cd43e1b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d4030119-6037-41c6-a17f-bc5e3cd43e1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:54.300392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:54.301948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:54.302788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:54.302788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:54.302788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:54.302788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:54.304941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 240/486] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:54.310124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4030119-6037-41c6-a17f-bc5e3cd43e1b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:54.325071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7e516302-8840-4357-aecf-e619fdd49d3b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:54.326284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7e516302-8840-4357-aecf-e619fdd49d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:54.368555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:54.406739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 247/487] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:54.430466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e516302-8840-4357-aecf-e619fdd49d3b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:54.484035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4030119-6037-41c6-a17f-bc5e3cd43e1b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:54.547660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccad1b1-ce53-4f46-8932-e5c9447a1060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:54.548309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccad1b1-ce53-4f46-8932-e5c9447a1060 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:54.548587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccad1b1-ce53-4f46-8932-e5c9447a1060 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:54.548857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccad1b1-ce53-4f46-8932-e5c9447a1060 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:54.549553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccad1b1-ce53-4f46-8932-e5c9447a1060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:54.549855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccad1b1-ce53-4f46-8932-e5c9447a1060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:54.552201 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccad1b1-ce53-4f46-8932-e5c9447a1060 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:54.570649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 241/488] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:54.580951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccad1b1-ce53-4f46-8932-e5c9447a1060 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:54.595733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e516302-8840-4357-aecf-e619fdd49d3b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:54.607492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:54.632607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5e080-6aee-4810-ac16-bdf049dac314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:54.633253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5e080-6aee-4810-ac16-bdf049dac314 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:54.633538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5e080-6aee-4810-ac16-bdf049dac314 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:54.633814 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5e080-6aee-4810-ac16-bdf049dac314 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:54.645167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5e080-6aee-4810-ac16-bdf049dac314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3bc5e080-6aee-4810-ac16-bdf049dac314 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:54.651145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:54.652076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:54.652870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 248/489] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:23:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:54.664610 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5e080-6aee-4810-ac16-bdf049dac314 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:54.773190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccad1b1-ce53-4f46-8932-e5c9447a1060 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:54.782803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:54.813881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bec44-5c6f-4b82-826f-c9c8286c2333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:54.814440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bec44-5c6f-4b82-826f-c9c8286c2333 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:54.814721 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bec44-5c6f-4b82-826f-c9c8286c2333 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:54.814995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bec44-5c6f-4b82-826f-c9c8286c2333 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:54.822880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193bec44-5c6f-4b82-826f-c9c8286c2333 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:54.824096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5e080-6aee-4810-ac16-bdf049dac314 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:54.824357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193bec44-5c6f-4b82-826f-c9c8286c2333 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-193bec44-5c6f-4b82-826f-c9c8286c2333 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:23:54.837162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:54.838280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:54.839335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:54.839335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:54.839335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:23:54.839335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:23:54.839335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:23:54.839335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:23:54.839335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:23:54.839335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:23:54.839335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:23:54.839335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:54.839335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 242/490] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Aug 4 07:23:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:54.840023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7b870-06b7-449a-b182-c93f0492f3fe None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:54.840207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7b870-06b7-449a-b182-c93f0492f3fe None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:54.840485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7b870-06b7-449a-b182-c93f0492f3fe None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:54.842285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 243/491] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:54.844469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7b870-06b7-449a-b182-c93f0492f3fe None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:54.870695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158e2cd-2adf-4220-a4c7-8b2374f13a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:54.871598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e2cd-2adf-4220-a4c7-8b2374f13a90 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:54.871888 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e2cd-2adf-4220-a4c7-8b2374f13a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:54.872193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e2cd-2adf-4220-a4c7-8b2374f13a90 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:54.872917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158e2cd-2adf-4220-a4c7-8b2374f13a90 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:54.873482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158e2cd-2adf-4220-a4c7-8b2374f13a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:54.875588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158e2cd-2adf-4220-a4c7-8b2374f13a90 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:54.892943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 249/492] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:23:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:54.892943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c95c0f-31af-4b64-85b1-e6214a08653a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:54.892943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c95c0f-31af-4b64-85b1-e6214a08653a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:54.892943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c95c0f-31af-4b64-85b1-e6214a08653a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:54.914287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-06c95c0f-31af-4b64-85b1-e6214a08653a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:54.915733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-06c95c0f-31af-4b64-85b1-e6214a08653a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:54.944865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:54.970248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7b870-06b7-449a-b182-c93f0492f3fe None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:54.993926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 250/493] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:54.995048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c95c0f-31af-4b64-85b1-e6214a08653a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:55.026216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11caee3-4f20-47ef-91d7-d2ca91e4e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:55.026821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11caee3-4f20-47ef-91d7-d2ca91e4e4bb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.027108 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11caee3-4f20-47ef-91d7-d2ca91e4e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.027388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11caee3-4f20-47ef-91d7-d2ca91e4e4bb None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.028219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11caee3-4f20-47ef-91d7-d2ca91e4e4bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:55.028527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11caee3-4f20-47ef-91d7-d2ca91e4e4bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:55.030283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11caee3-4f20-47ef-91d7-d2ca91e4e4bb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:55.068561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 244/494] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Aug 4 07:23:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:55.069816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9b006-d7f4-417c-95bc-1425f2a2d0dc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.070107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9b006-d7f4-417c-95bc-1425f2a2d0dc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.070385 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9b006-d7f4-417c-95bc-1425f2a2d0dc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.071739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 245/495] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:55.072464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9b006-d7f4-417c-95bc-1425f2a2d0dc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:55.094899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:55.100413 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c95c0f-31af-4b64-85b1-e6214a08653a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:55.138048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f9759-04ab-4cdd-9e6e-bf9f1ad0daec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:55.138689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f9759-04ab-4cdd-9e6e-bf9f1ad0daec None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.138970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f9759-04ab-4cdd-9e6e-bf9f1ad0daec None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.139245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f9759-04ab-4cdd-9e6e-bf9f1ad0daec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.151116 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f9759-04ab-4cdd-9e6e-bf9f1ad0daec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:55.152731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f9759-04ab-4cdd-9e6e-bf9f1ad0daec None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e78f9759-04ab-4cdd-9e6e-bf9f1ad0daec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:55.169216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:55.170074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:55.170885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:55.170885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:55.170885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:55.170885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:55.170885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 251/496] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:55.171560 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f9759-04ab-4cdd-9e6e-bf9f1ad0daec None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:55.182080 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9b006-d7f4-417c-95bc-1425f2a2d0dc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:55.188274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:55.220293 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a165d13-710a-4e0b-96df-10c77ed510a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:55.221547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a165d13-710a-4e0b-96df-10c77ed510a6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.221982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a165d13-710a-4e0b-96df-10c77ed510a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.222267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a165d13-710a-4e0b-96df-10c77ed510a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.232598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a165d13-710a-4e0b-96df-10c77ed510a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8a165d13-710a-4e0b-96df-10c77ed510a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:55.237736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:55.239141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 246/497] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:23:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:55.240908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dff421-285f-4c19-89a7-158463bb3f89 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.241630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dff421-285f-4c19-89a7-158463bb3f89 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.241630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dff421-285f-4c19-89a7-158463bb3f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.259171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e4dff421-285f-4c19-89a7-158463bb3f89 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:55.260395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e4dff421-285f-4c19-89a7-158463bb3f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:55.286415 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f9759-04ab-4cdd-9e6e-bf9f1ad0daec None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:55.323691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66193613-3d9f-4958-8a2a-8040158a2e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:55.324258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66193613-3d9f-4958-8a2a-8040158a2e6c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.324552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66193613-3d9f-4958-8a2a-8040158a2e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.324822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66193613-3d9f-4958-8a2a-8040158a2e6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.331097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 247/498] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:55.332470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dff421-285f-4c19-89a7-158463bb3f89 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:55.346471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:55.405553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66193613-3d9f-4958-8a2a-8040158a2e6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:55.407019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66193613-3d9f-4958-8a2a-8040158a2e6c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:55.421531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:55.444559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dff421-285f-4c19-89a7-158463bb3f89 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:55.485941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bf650-2626-426f-89b0-b25a0462347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:55.486571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf650-2626-426f-89b0-b25a0462347f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.486854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf650-2626-426f-89b0-b25a0462347f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.487130 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf650-2626-426f-89b0-b25a0462347f None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.487823 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bf650-2626-426f-89b0-b25a0462347f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:55.488799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bf650-2626-426f-89b0-b25a0462347f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:55.490275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bf650-2626-426f-89b0-b25a0462347f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:55.502375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 248/499] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:55.503048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bf650-2626-426f-89b0-b25a0462347f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:55.520955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 252/500] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:55.521800 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66193613-3d9f-4958-8a2a-8040158a2e6c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:55.578470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:55.610779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bf650-2626-426f-89b0-b25a0462347f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:55.627647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66193613-3d9f-4958-8a2a-8040158a2e6c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:55.642074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:55.647960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c8b19-2603-4d25-af3b-aa2984d77799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:55.649114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c8b19-2603-4d25-af3b-aa2984d77799 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.655649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c8b19-2603-4d25-af3b-aa2984d77799 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.655649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c8b19-2603-4d25-af3b-aa2984d77799 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.655649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c8b19-2603-4d25-af3b-aa2984d77799 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:55.655649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c8b19-2603-4d25-af3b-aa2984d77799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:55.655649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c8b19-2603-4d25-af3b-aa2984d77799 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:55.665726 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 249/501] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:23:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:55.665726 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c8b19-2603-4d25-af3b-aa2984d77799 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:55.667288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97194429-ee13-4f09-a471-215ecd2ad4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:55.667941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97194429-ee13-4f09-a471-215ecd2ad4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.668248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97194429-ee13-4f09-a471-215ecd2ad4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.668519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97194429-ee13-4f09-a471-215ecd2ad4f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.678891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97194429-ee13-4f09-a471-215ecd2ad4f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-97194429-ee13-4f09-a471-215ecd2ad4f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:55.684178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:55.685070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:55.685860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 253/502] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:23:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:55.686585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97194429-ee13-4f09-a471-215ecd2ad4f5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:55.770209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c8b19-2603-4d25-af3b-aa2984d77799 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:55.785641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97194429-ee13-4f09-a471-215ecd2ad4f5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:55.801944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:55.807185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44b227-a4e8-4468-b491-8fabd3122b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:55.807819 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44b227-a4e8-4468-b491-8fabd3122b7d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.808132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44b227-a4e8-4468-b491-8fabd3122b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.808652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44b227-a4e8-4468-b491-8fabd3122b7d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.815998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44b227-a4e8-4468-b491-8fabd3122b7d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:55.818091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44b227-a4e8-4468-b491-8fabd3122b7d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:55.819794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f00b56-4c77-42c3-884d-1b16d11baea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:55.821604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00b56-4c77-42c3-884d-1b16d11baea9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.821604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00b56-4c77-42c3-884d-1b16d11baea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.821604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00b56-4c77-42c3-884d-1b16d11baea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3d44b227-a4e8-4468-b491-8fabd3122b7d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:23:55.829200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:55.830107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:55.831005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:55.831005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:55.831005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:23:55.831005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:23:55.831005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:23:55.831005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:23:55.831005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:23:55.831005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:23:55.831005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 07:23:55.831005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:55.831005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 250/503] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:55.831563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44b227-a4e8-4468-b491-8fabd3122b7d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:55.836681 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f00b56-4c77-42c3-884d-1b16d11baea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:55.838167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f00b56-4c77-42c3-884d-1b16d11baea9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-39f00b56-4c77-42c3-884d-1b16d11baea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:55.852690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:55.853695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:55.854559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:55.854559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:55.854559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:55.854559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:55.856140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 254/504] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:55.857128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f00b56-4c77-42c3-884d-1b16d11baea9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:55.884311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:55.931244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44b227-a4e8-4468-b491-8fabd3122b7d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:55.956145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f00b56-4c77-42c3-884d-1b16d11baea9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:55.969893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6fdea-c2c2-4723-87fe-40ca1fd082c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:55.970542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6fdea-c2c2-4723-87fe-40ca1fd082c7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.970819 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6fdea-c2c2-4723-87fe-40ca1fd082c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.971095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6fdea-c2c2-4723-87fe-40ca1fd082c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.982812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6fdea-c2c2-4723-87fe-40ca1fd082c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:55.984584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6fdea-c2c2-4723-87fe-40ca1fd082c7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:55.991449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c4b9d-6329-46d5-859a-5464f4ba2ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:55.992084 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c4b9d-6329-46d5-859a-5464f4ba2ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:55.992370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c4b9d-6329-46d5-859a-5464f4ba2ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:55.992645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c4b9d-6329-46d5-859a-5464f4ba2ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:55.993350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c4b9d-6329-46d5-859a-5464f4ba2ec7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:55.993657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c4b9d-6329-46d5-859a-5464f4ba2ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:55.995039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c4b9d-6329-46d5-859a-5464f4ba2ec7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e2c6fdea-c2c2-4723-87fe-40ca1fd082c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:55.999037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:55.999942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:56.000805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:56.000805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:56.000805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:56.000805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:56.000805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 251/505] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:56.001516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7046-e816-4956-b424-9ecda8017a5a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.001753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7046-e816-4956-b424-9ecda8017a5a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.002037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7046-e816-4956-b424-9ecda8017a5a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.008144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 252/506] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:56.008144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d7046-e816-4956-b424-9ecda8017a5a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.008368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 255/507] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:23:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:56.010376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c4b9d-6329-46d5-859a-5464f4ba2ec7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.054182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:56.098545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:56.107298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d7046-e816-4956-b424-9ecda8017a5a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:56.110253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c4b9d-6329-46d5-859a-5464f4ba2ec7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:56.141240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed810fb-1f7f-4f8f-b7e8-df6d743a37a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:56.141855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed810fb-1f7f-4f8f-b7e8-df6d743a37a8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.142138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed810fb-1f7f-4f8f-b7e8-df6d743a37a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.142414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed810fb-1f7f-4f8f-b7e8-df6d743a37a8 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.143207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed810fb-1f7f-4f8f-b7e8-df6d743a37a8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:56.143514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed810fb-1f7f-4f8f-b7e8-df6d743a37a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:56.144632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527a006-163d-48ae-aa85-2faa1ed92f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:56.145137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed810fb-1f7f-4f8f-b7e8-df6d743a37a8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:56.145239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527a006-163d-48ae-aa85-2faa1ed92f11 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.145505 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527a006-163d-48ae-aa85-2faa1ed92f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.145781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527a006-163d-48ae-aa85-2faa1ed92f11 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.146470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527a006-163d-48ae-aa85-2faa1ed92f11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:56.146774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527a006-163d-48ae-aa85-2faa1ed92f11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:56.148148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527a006-163d-48ae-aa85-2faa1ed92f11 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:56.159031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 256/508] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:56.159808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527a006-163d-48ae-aa85-2faa1ed92f11 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.170052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 253/509] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:56.170052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed810fb-1f7f-4f8f-b7e8-df6d743a37a8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.259084 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527a006-163d-48ae-aa85-2faa1ed92f11 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:56.266296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed810fb-1f7f-4f8f-b7e8-df6d743a37a8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:56.267805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:56.293299 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4890a6-8206-4a6f-80ff-b102b4e9bd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:56.293912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4890a6-8206-4a6f-80ff-b102b4e9bd2e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.294193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4890a6-8206-4a6f-80ff-b102b4e9bd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.294516 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4890a6-8206-4a6f-80ff-b102b4e9bd2e None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.295210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4890a6-8206-4a6f-80ff-b102b4e9bd2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:56.295517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4890a6-8206-4a6f-80ff-b102b4e9bd2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:56.296930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4890a6-8206-4a6f-80ff-b102b4e9bd2e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:56.301266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a6987-25dc-4e64-a19b-85c440290d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:56.301894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a6987-25dc-4e64-a19b-85c440290d9a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.302180 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a6987-25dc-4e64-a19b-85c440290d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.302456 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a6987-25dc-4e64-a19b-85c440290d9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.308825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 257/510] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:56.309966 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3671e0-ce84-48f8-b720-7f7328ba6310 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.310247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3671e0-ce84-48f8-b720-7f7328ba6310 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.310520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3671e0-ce84-48f8-b720-7f7328ba6310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.318100 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a6987-25dc-4e64-a19b-85c440290d9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:56.319551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a6987-25dc-4e64-a19b-85c440290d9a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:56.327736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5e3671e0-ce84-48f8-b720-7f7328ba6310 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:56.328931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5e3671e0-ce84-48f8-b720-7f7328ba6310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:56.330823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-910a6987-25dc-4e64-a19b-85c440290d9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:56.334860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:56.340327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:56.341290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:56.341290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:56.341290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:56.341290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:56.341290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 254/511] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:56.341290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a6987-25dc-4e64-a19b-85c440290d9a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.398213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 258/512] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:56.398971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3671e0-ce84-48f8-b720-7f7328ba6310 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.454013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a6987-25dc-4e64-a19b-85c440290d9a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:56.487477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595446bb-778a-4dd8-af3a-450b6ed91993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:56.488678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595446bb-778a-4dd8-af3a-450b6ed91993 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.488967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595446bb-778a-4dd8-af3a-450b6ed91993 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.489418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595446bb-778a-4dd8-af3a-450b6ed91993 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.495572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3671e0-ce84-48f8-b720-7f7328ba6310 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:56.496038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595446bb-778a-4dd8-af3a-450b6ed91993 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:56.497483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595446bb-778a-4dd8-af3a-450b6ed91993 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-595446bb-778a-4dd8-af3a-450b6ed91993 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:23:56.509653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:56.510477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:56.511322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:23:56.511322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:23:56.511322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:23:56.511322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:23:56.511322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:23:56.511322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:23:56.511322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:23:56.511322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:56.511322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 255/513] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:56.511802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595446bb-778a-4dd8-af3a-450b6ed91993 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.528569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:56.530986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b38269-b23d-4285-97f5-7f177440f842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:56.531603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b38269-b23d-4285-97f5-7f177440f842 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.531886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b38269-b23d-4285-97f5-7f177440f842 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.532200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b38269-b23d-4285-97f5-7f177440f842 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.540950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:56.599077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b38269-b23d-4285-97f5-7f177440f842 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:56.600591 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b38269-b23d-4285-97f5-7f177440f842 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:56.611467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595446bb-778a-4dd8-af3a-450b6ed91993 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:56.646367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f995b47-0d81-41fa-86df-c804d90a47d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:56.646993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f995b47-0d81-41fa-86df-c804d90a47d1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.647281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f995b47-0d81-41fa-86df-c804d90a47d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.647555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f995b47-0d81-41fa-86df-c804d90a47d1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.654773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f995b47-0d81-41fa-86df-c804d90a47d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:56.656197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f995b47-0d81-41fa-86df-c804d90a47d1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0f995b47-0d81-41fa-86df-c804d90a47d1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:23:56.668405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:56.669510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:56.670538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:56.670538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:56.670538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:23:56.670538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:23:56.670538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:23:56.670538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:23:56.670538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:23:56.670538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:23:56.670538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 07:23:56.670538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:56.672125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 256/514] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:56.673074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f995b47-0d81-41fa-86df-c804d90a47d1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.696445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 259/515] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:56.697274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b38269-b23d-4285-97f5-7f177440f842 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.735016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:56.750833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:56.775345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f995b47-0d81-41fa-86df-c804d90a47d1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:56.794559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b38269-b23d-4285-97f5-7f177440f842 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:56.808819 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b8d52-e770-4e35-8839-c6373bae146d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:56.809506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b8d52-e770-4e35-8839-c6373bae146d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.809876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b8d52-e770-4e35-8839-c6373bae146d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.810217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b8d52-e770-4e35-8839-c6373bae146d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.810973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b8d52-e770-4e35-8839-c6373bae146d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:56.811280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b8d52-e770-4e35-8839-c6373bae146d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:56.812665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b8d52-e770-4e35-8839-c6373bae146d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:56.824102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 257/516] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:56.824851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b8d52-e770-4e35-8839-c6373bae146d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.827647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae185e07-3467-417f-8723-7f9055f4cfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:56.828315 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185e07-3467-417f-8723-7f9055f4cfee None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.828663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185e07-3467-417f-8723-7f9055f4cfee None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.828878 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185e07-3467-417f-8723-7f9055f4cfee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.839643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae185e07-3467-417f-8723-7f9055f4cfee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:56.841131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae185e07-3467-417f-8723-7f9055f4cfee None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ae185e07-3467-417f-8723-7f9055f4cfee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:56.856421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:56.857316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:56.858412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:56.858412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:56.858412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:56.858412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:56.858412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 260/517] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:56.858674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae185e07-3467-417f-8723-7f9055f4cfee None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.927056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b8d52-e770-4e35-8839-c6373bae146d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:56.947203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:56.960246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae185e07-3467-417f-8723-7f9055f4cfee None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:56.961229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d8811-bd1a-432c-b601-1ab24e2e0b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:56.961855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d8811-bd1a-432c-b601-1ab24e2e0b19 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.962133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d8811-bd1a-432c-b601-1ab24e2e0b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.962418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d8811-bd1a-432c-b601-1ab24e2e0b19 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.963169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d8811-bd1a-432c-b601-1ab24e2e0b19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:56.963477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d8811-bd1a-432c-b601-1ab24e2e0b19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:56.964941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d8811-bd1a-432c-b601-1ab24e2e0b19 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:56.966652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:56.982950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 258/518] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:56.984031 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d8811-bd1a-432c-b601-1ab24e2e0b19 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:56.995290 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1634880-341e-4ab8-87ce-2faa7c577731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:56.995953 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1634880-341e-4ab8-87ce-2faa7c577731 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:56.996251 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1634880-341e-4ab8-87ce-2faa7c577731 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:56.996530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1634880-341e-4ab8-87ce-2faa7c577731 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:56.997325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1634880-341e-4ab8-87ce-2faa7c577731 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:56.997631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1634880-341e-4ab8-87ce-2faa7c577731 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:56.999082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1634880-341e-4ab8-87ce-2faa7c577731 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:57.035785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 261/519] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:57.036640 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1634880-341e-4ab8-87ce-2faa7c577731 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:57.085348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d8811-bd1a-432c-b601-1ab24e2e0b19 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:57.118969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b0165-829c-4171-b1ef-86943fc775ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:57.119603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b0165-829c-4171-b1ef-86943fc775ee None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:57.119883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b0165-829c-4171-b1ef-86943fc775ee None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:57.120193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b0165-829c-4171-b1ef-86943fc775ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:57.130374 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b0165-829c-4171-b1ef-86943fc775ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:57.131824 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b0165-829c-4171-b1ef-86943fc775ee None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:57.135467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1634880-341e-4ab8-87ce-2faa7c577731 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-135b0165-829c-4171-b1ef-86943fc775ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:57.146226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:57.147135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:57.148039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:57.148039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:57.148039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:57.148039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:57.148039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 259/520] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:57.148301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b0165-829c-4171-b1ef-86943fc775ee None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:57.170005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315e63b-096d-4a0f-a46d-66f3f8f3bd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:57.170719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315e63b-096d-4a0f-a46d-66f3f8f3bd66 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:57.171010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315e63b-096d-4a0f-a46d-66f3f8f3bd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:57.171285 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315e63b-096d-4a0f-a46d-66f3f8f3bd66 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:57.178816 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315e63b-096d-4a0f-a46d-66f3f8f3bd66 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:57.180231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315e63b-096d-4a0f-a46d-66f3f8f3bd66 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5315e63b-096d-4a0f-a46d-66f3f8f3bd66 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:23:57.193203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:57.194221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 262/521] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:23:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:57.195110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315e63b-096d-4a0f-a46d-66f3f8f3bd66 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:57.201660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:57.205843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:57.255993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b0165-829c-4171-b1ef-86943fc775ee None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:57.289461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3d1a8-ff16-4303-a762-7aac940796fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:57.290080 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3d1a8-ff16-4303-a762-7aac940796fd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:57.290363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3d1a8-ff16-4303-a762-7aac940796fd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:57.290645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3d1a8-ff16-4303-a762-7aac940796fd None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:57.291341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3d1a8-ff16-4303-a762-7aac940796fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:57.291645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3d1a8-ff16-4303-a762-7aac940796fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:57.293042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3d1a8-ff16-4303-a762-7aac940796fd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:57.293389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315e63b-096d-4a0f-a46d-66f3f8f3bd66 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:57.304651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 260/522] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:57.305516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3d1a8-ff16-4303-a762-7aac940796fd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:57.327488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226dd46-80ee-4883-a635-d3cfdb92b329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:57.328131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226dd46-80ee-4883-a635-d3cfdb92b329 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:57.328417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226dd46-80ee-4883-a635-d3cfdb92b329 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:57.328697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226dd46-80ee-4883-a635-d3cfdb92b329 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:57.329486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226dd46-80ee-4883-a635-d3cfdb92b329 None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=4eedd711c6114de1bfcdf692ce058794) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:57.329802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226dd46-80ee-4883-a635-d3cfdb92b329 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:57.331240 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226dd46-80ee-4883-a635-d3cfdb92b329 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:57.336847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 263/523] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:57 2022] GET /identity/v3/role_assignments?user.id=4eedd711c6114de1bfcdf692ce058794&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:57.337615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226dd46-80ee-4883-a635-d3cfdb92b329 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:57.417838 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3d1a8-ff16-4303-a762-7aac940796fd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:57.420430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:57.426939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:57.455879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226dd46-80ee-4883-a635-d3cfdb92b329 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:57.456389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c237f42-11f2-415f-bc76-a80bc3ca9aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:57.457704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c237f42-11f2-415f-bc76-a80bc3ca9aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:57.457982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c237f42-11f2-415f-bc76-a80bc3ca9aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:57.458273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c237f42-11f2-415f-bc76-a80bc3ca9aa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:57.497658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9ca1d-390c-4bbb-8941-4ec6243e444a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:57.498277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9ca1d-390c-4bbb-8941-4ec6243e444a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:57.498562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9ca1d-390c-4bbb-8941-4ec6243e444a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:57.498838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9ca1d-390c-4bbb-8941-4ec6243e444a None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:57.499543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9ca1d-390c-4bbb-8941-4ec6243e444a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:57.499853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9ca1d-390c-4bbb-8941-4ec6243e444a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:57.508317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9ca1d-390c-4bbb-8941-4ec6243e444a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:57.518111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 264/524] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:57.518111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9ca1d-390c-4bbb-8941-4ec6243e444a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:57.549589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c237f42-11f2-415f-bc76-a80bc3ca9aa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:57.552573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c237f42-11f2-415f-bc76-a80bc3ca9aa3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:57.618937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9ca1d-390c-4bbb-8941-4ec6243e444a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:57.650792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:57.655750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7109f-a76e-496f-92b1-e337482ac297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:57.656356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7109f-a76e-496f-92b1-e337482ac297 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:57.656642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7109f-a76e-496f-92b1-e337482ac297 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:57.657050 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7109f-a76e-496f-92b1-e337482ac297 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:57.657713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec7109f-a76e-496f-92b1-e337482ac297 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:57.658024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec7109f-a76e-496f-92b1-e337482ac297 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:57.659485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec7109f-a76e-496f-92b1-e337482ac297 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:57.668428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 261/525] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:57.668428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c237f42-11f2-415f-bc76-a80bc3ca9aa3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:57.675058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:57.686406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 265/526] 10.222.0.64 () {56 vars in 1139 bytes} [Thu Aug 4 07:23:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:57.687371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7109f-a76e-496f-92b1-e337482ac297 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:57.771111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c237f42-11f2-415f-bc76-a80bc3ca9aa3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:57.790771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7109f-a76e-496f-92b1-e337482ac297 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:57.809013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42707c99-fbbb-4776-bc66-f300d3bee849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:57.809642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42707c99-fbbb-4776-bc66-f300d3bee849 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:57.809907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42707c99-fbbb-4776-bc66-f300d3bee849 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:57.810179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42707c99-fbbb-4776-bc66-f300d3bee849 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:57.810862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42707c99-fbbb-4776-bc66-f300d3bee849 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:57.812323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42707c99-fbbb-4776-bc66-f300d3bee849 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:57.825815 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42126819-df0d-4be8-ac92-1b11d49074bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:57.826470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42126819-df0d-4be8-ac92-1b11d49074bc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:57.826757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42126819-df0d-4be8-ac92-1b11d49074bc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:57.827034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42126819-df0d-4be8-ac92-1b11d49074bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:57.833797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42126819-df0d-4be8-ac92-1b11d49074bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:57.835245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42126819-df0d-4be8-ac92-1b11d49074bc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-42126819-df0d-4be8-ac92-1b11d49074bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:23:57.846957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:57.848678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:57.849573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:23:57.849573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:23:57.849573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:23:57.849573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:23:57.849573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:23:57.849573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:23:57.849573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:23:57.849573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:57.849573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 266/527] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:57.849573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42126819-df0d-4be8-ac92-1b11d49074bc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:57.887932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 262/528] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:23:57 2022] POST /identity/v3/users => generated 247 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:23:57.901133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42707c99-fbbb-4776-bc66-f300d3bee849 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:57.902950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:57.992262 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42126819-df0d-4be8-ac92-1b11d49074bc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:57.993181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:58.041750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536a6d5-88fc-45fe-a7d4-1b9279f0e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:58.042513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a6d5-88fc-45fe-a7d4-1b9279f0e975 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:58.042965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a6d5-88fc-45fe-a7d4-1b9279f0e975 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:58.043273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a6d5-88fc-45fe-a7d4-1b9279f0e975 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:58.052947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42707c99-fbbb-4776-bc66-f300d3bee849 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:58.056296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536a6d5-88fc-45fe-a7d4-1b9279f0e975 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:58.058594 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536a6d5-88fc-45fe-a7d4-1b9279f0e975 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f536a6d5-88fc-45fe-a7d4-1b9279f0e975 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:58.076797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:58.078009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:58.078930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:58.078930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:58.078930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:58.078930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:58.081012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 267/529] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:58.081012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536a6d5-88fc-45fe-a7d4-1b9279f0e975 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:58.104263 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988044f-5fde-4fab-b820-cc8d12c099db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:58.105433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988044f-5fde-4fab-b820-cc8d12c099db None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:58.106176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988044f-5fde-4fab-b820-cc8d12c099db None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:58.106637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988044f-5fde-4fab-b820-cc8d12c099db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:58.197462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536a6d5-88fc-45fe-a7d4-1b9279f0e975 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:58.197957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988044f-5fde-4fab-b820-cc8d12c099db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:58.199378 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:58.201922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988044f-5fde-4fab-b820-cc8d12c099db None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:58.234071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabae58-ba34-4991-b663-e68bc2aed076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:58.234928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabae58-ba34-4991-b663-e68bc2aed076 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:58.235358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabae58-ba34-4991-b663-e68bc2aed076 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:58.235771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabae58-ba34-4991-b663-e68bc2aed076 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:58.236712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabae58-ba34-4991-b663-e68bc2aed076 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:58.237164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabae58-ba34-4991-b663-e68bc2aed076 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:58.239375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabae58-ba34-4991-b663-e68bc2aed076 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:58.263254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 268/530] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:23:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:58.264406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabae58-ba34-4991-b663-e68bc2aed076 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:58.271874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:58.313704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 263/531] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:23:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:23:58.314685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988044f-5fde-4fab-b820-cc8d12c099db None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:58.369430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabae58-ba34-4991-b663-e68bc2aed076 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:58.405285 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c5b6a-9551-4e30-8bd2-ce1f5f1a459c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:58.406145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c5b6a-9551-4e30-8bd2-ce1f5f1a459c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:58.406595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c5b6a-9551-4e30-8bd2-ce1f5f1a459c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:58.407012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c5b6a-9551-4e30-8bd2-ce1f5f1a459c None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:58.407847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c5b6a-9551-4e30-8bd2-ce1f5f1a459c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:58.408318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c5b6a-9551-4e30-8bd2-ce1f5f1a459c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:58.409983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c5b6a-9551-4e30-8bd2-ce1f5f1a459c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:58.421820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988044f-5fde-4fab-b820-cc8d12c099db None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:58.422377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 269/532] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:58.423459 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c5b6a-9551-4e30-8bd2-ce1f5f1a459c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:58.456449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:58.460995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f606613-ee7d-4e25-88f7-70a7e48b7c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:58.462627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f606613-ee7d-4e25-88f7-70a7e48b7c5c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:58.462627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f606613-ee7d-4e25-88f7-70a7e48b7c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:58.462627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f606613-ee7d-4e25-88f7-70a7e48b7c5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:58.463392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f606613-ee7d-4e25-88f7-70a7e48b7c5c None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=eab096d4ccae4bc09e6177afd85a55de) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:58.463792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f606613-ee7d-4e25-88f7-70a7e48b7c5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:58.465325 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f606613-ee7d-4e25-88f7-70a7e48b7c5c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:58.473656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 264/533] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:58 2022] GET /identity/v3/role_assignments?user.id=eab096d4ccae4bc09e6177afd85a55de&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:58.474663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f606613-ee7d-4e25-88f7-70a7e48b7c5c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:58.506001 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:58.535883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c5b6a-9551-4e30-8bd2-ce1f5f1a459c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:58.594497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f606613-ee7d-4e25-88f7-70a7e48b7c5c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:58.634640 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1e59a-fe67-402b-8857-b757adc2dd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:58.635698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1e59a-fe67-402b-8857-b757adc2dd07 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:58.636175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1e59a-fe67-402b-8857-b757adc2dd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:58.636737 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1e59a-fe67-402b-8857-b757adc2dd07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:58.644946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1e59a-fe67-402b-8857-b757adc2dd07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:58.646712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1e59a-fe67-402b-8857-b757adc2dd07 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:58.652062 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb302d-0722-471a-aa7b-88924eaee042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:58.652738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb302d-0722-471a-aa7b-88924eaee042 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:58.653024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb302d-0722-471a-aa7b-88924eaee042 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:58.653303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb302d-0722-471a-aa7b-88924eaee042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6da1e59a-fe67-402b-8857-b757adc2dd07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:23:58.661168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:58.662283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:58.663321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:23:58.663321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:23:58.663321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:23:58.663321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:23:58.663321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:23:58.663321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:23:58.663321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:23:58.663321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:58.665758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 265/534] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:58.670989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1e59a-fe67-402b-8857-b757adc2dd07 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:58.688605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb302d-0722-471a-aa7b-88924eaee042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:58.690079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb302d-0722-471a-aa7b-88924eaee042 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-97eb302d-0722-471a-aa7b-88924eaee042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:23:58.737552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:58.738808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:58.740065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:23:58.740065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:23:58.740065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:23:58.740065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:58.740065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 270/535] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:58.744888 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb302d-0722-471a-aa7b-88924eaee042 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:58.776389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:58.788669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1e59a-fe67-402b-8857-b757adc2dd07 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:58.827527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0505f2-3d9d-42b9-9a6a-1a920ab6dc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:58.828902 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0505f2-3d9d-42b9-9a6a-1a920ab6dc76 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:58.829386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0505f2-3d9d-42b9-9a6a-1a920ab6dc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:58.829824 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0505f2-3d9d-42b9-9a6a-1a920ab6dc76 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:58.830792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0505f2-3d9d-42b9-9a6a-1a920ab6dc76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:58.831265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0505f2-3d9d-42b9-9a6a-1a920ab6dc76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:58.834306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0505f2-3d9d-42b9-9a6a-1a920ab6dc76 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:58.850089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 266/536] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:23:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:58.855347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0505f2-3d9d-42b9-9a6a-1a920ab6dc76 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:58.950053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:58.980250 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0505f2-3d9d-42b9-9a6a-1a920ab6dc76 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:59.009155 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb302d-0722-471a-aa7b-88924eaee042 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:59.024547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d633270-ff76-4c0b-bc4a-b06a9a479e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:59.025411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d633270-ff76-4c0b-bc4a-b06a9a479e19 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:59.025841 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d633270-ff76-4c0b-bc4a-b06a9a479e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:59.026677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d633270-ff76-4c0b-bc4a-b06a9a479e19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:59.027767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d633270-ff76-4c0b-bc4a-b06a9a479e19 None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=56dfc4dfda424d40b7c9f17345b8ad45) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:59.028403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d633270-ff76-4c0b-bc4a-b06a9a479e19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:59.030324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d633270-ff76-4c0b-bc4a-b06a9a479e19 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:59.037501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 267/537] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:58 2022] GET /identity/v3/role_assignments?user.id=56dfc4dfda424d40b7c9f17345b8ad45&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:59.060365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4550db-da08-487d-b117-618b76c979f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:59.061174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4550db-da08-487d-b117-618b76c979f5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:59.061555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4550db-da08-487d-b117-618b76c979f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:59.061931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4550db-da08-487d-b117-618b76c979f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:59.062750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4550db-da08-487d-b117-618b76c979f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:59.063162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4550db-da08-487d-b117-618b76c979f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:59.064859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4550db-da08-487d-b117-618b76c979f5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:59.079593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 271/538] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:59.085561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d633270-ff76-4c0b-bc4a-b06a9a479e19 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:59.110825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:59.231169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d633270-ff76-4c0b-bc4a-b06a9a479e19 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:59.266637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278a1e7-755f-4ac8-afe8-622adf566e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:59.267386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278a1e7-755f-4ac8-afe8-622adf566e41 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:59.267676 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278a1e7-755f-4ac8-afe8-622adf566e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:59.267996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278a1e7-755f-4ac8-afe8-622adf566e41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:59.275194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278a1e7-755f-4ac8-afe8-622adf566e41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:59.276677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278a1e7-755f-4ac8-afe8-622adf566e41 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6278a1e7-755f-4ac8-afe8-622adf566e41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:23:59.288280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:59.289152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:59.289979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:23:59.289979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:23:59.289979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:23:59.289979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:23:59.289979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:23:59.289979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:23:59.289979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:23:59.289979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:23:59.289979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 268/539] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:23:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:23:59.295049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4550db-da08-487d-b117-618b76c979f5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:59.404379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4550db-da08-487d-b117-618b76c979f5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:59.451587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19062e1c-5633-4f40-b4cf-d31f4ce5b45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:59.452229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19062e1c-5633-4f40-b4cf-d31f4ce5b45a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:59.452578 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19062e1c-5633-4f40-b4cf-d31f4ce5b45a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:59.452857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19062e1c-5633-4f40-b4cf-d31f4ce5b45a None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:59.453655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19062e1c-5633-4f40-b4cf-d31f4ce5b45a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:59.453969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19062e1c-5633-4f40-b4cf-d31f4ce5b45a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:59.455418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19062e1c-5633-4f40-b4cf-d31f4ce5b45a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:59.457258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:59.468908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 272/540] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:23:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:23:59.477067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278a1e7-755f-4ac8-afe8-622adf566e41 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:59.541562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b864c-3756-4c55-a470-20a9b28bb3b4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:59.542224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b864c-3756-4c55-a470-20a9b28bb3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:59.542653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b864c-3756-4c55-a470-20a9b28bb3b4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:59.545003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 273/541] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:23:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:23:59.550760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744138ee-3c4a-4c50-a690-ec000825494b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:59.551284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744138ee-3c4a-4c50-a690-ec000825494b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:59.551728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744138ee-3c4a-4c50-a690-ec000825494b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:59.573605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-744138ee-3c4a-4c50-a690-ec000825494b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:23:59.575549 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-744138ee-3c4a-4c50-a690-ec000825494b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:23:59.593857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:59.655903 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278a1e7-755f-4ac8-afe8-622adf566e41 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:59.658827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 274/542] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:23:59.697721 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744138ee-3c4a-4c50-a690-ec000825494b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:23:59.703865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af19335-140a-4c35-8623-a956866e54b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:59.704567 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af19335-140a-4c35-8623-a956866e54b7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:59.704851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af19335-140a-4c35-8623-a956866e54b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:59.705172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af19335-140a-4c35-8623-a956866e54b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:59.705980 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af19335-140a-4c35-8623-a956866e54b7 None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=b80e701a81cb4fa893e55288559010f5) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:23:59.706296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af19335-140a-4c35-8623-a956866e54b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:23:59.708021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af19335-140a-4c35-8623-a956866e54b7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:23:59.719955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 269/543] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:23:59 2022] GET /identity/v3/role_assignments?user.id=b80e701a81cb4fa893e55288559010f5&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:23:59.856218 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744138ee-3c4a-4c50-a690-ec000825494b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:23:59.883362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:23:59.903779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61122c3a-d1c9-4001-9f7c-cd1cdeee4930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:23:59.904487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61122c3a-d1c9-4001-9f7c-cd1cdeee4930 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:23:59.904772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61122c3a-d1c9-4001-9f7c-cd1cdeee4930 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:23:59.905054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61122c3a-d1c9-4001-9f7c-cd1cdeee4930 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:23:59.923647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61122c3a-d1c9-4001-9f7c-cd1cdeee4930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-61122c3a-d1c9-4001-9f7c-cd1cdeee4930 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:23:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:23:59.930496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:23:59.931292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 275/544] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:23:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:23:59.937280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af19335-140a-4c35-8623-a956866e54b7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:00.083109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af19335-140a-4c35-8623-a956866e54b7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:00.123048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f20596-70c0-4843-b128-9f1a194f16ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:00.123672 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f20596-70c0-4843-b128-9f1a194f16ae None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:00.123959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f20596-70c0-4843-b128-9f1a194f16ae None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:00.124265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f20596-70c0-4843-b128-9f1a194f16ae None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:00.124987 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f20596-70c0-4843-b128-9f1a194f16ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:00.125293 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f20596-70c0-4843-b128-9f1a194f16ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:00.126665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f20596-70c0-4843-b128-9f1a194f16ae None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:00.128031 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:00.140988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 270/545] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:23:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:00.147294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61122c3a-d1c9-4001-9f7c-cd1cdeee4930 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:00.340844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61122c3a-d1c9-4001-9f7c-cd1cdeee4930 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:00.382926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240b007-cb4f-431f-94c5-49334ddae928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:00.383573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240b007-cb4f-431f-94c5-49334ddae928 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:00.383859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240b007-cb4f-431f-94c5-49334ddae928 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:00.384160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240b007-cb4f-431f-94c5-49334ddae928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:00.399729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240b007-cb4f-431f-94c5-49334ddae928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:00.401247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240b007-cb4f-431f-94c5-49334ddae928 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e240b007-cb4f-431f-94c5-49334ddae928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:00.419884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:00.420785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:00.421619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:00.421619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:00.421619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:00.421619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:00.421619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 276/546] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:00.426872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f20596-70c0-4843-b128-9f1a194f16ae None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:00.551962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f20596-70c0-4843-b128-9f1a194f16ae None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:00.597496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a6a33-628b-469f-a2fc-bbaed4301fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:00.598294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a6a33-628b-469f-a2fc-bbaed4301fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:00.598581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a6a33-628b-469f-a2fc-bbaed4301fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:00.598854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a6a33-628b-469f-a2fc-bbaed4301fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:00.599685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a6a33-628b-469f-a2fc-bbaed4301fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:00.600007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a6a33-628b-469f-a2fc-bbaed4301fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:00.601390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a6a33-628b-469f-a2fc-bbaed4301fb0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:00.614651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 271/547] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:00.619356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12c2a4-bf78-494a-ab0e-c90da9b2b77c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:00.619903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12c2a4-bf78-494a-ab0e-c90da9b2b77c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:00.620411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12c2a4-bf78-494a-ab0e-c90da9b2b77c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:00.620907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a6a33-628b-469f-a2fc-bbaed4301fb0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:00.622237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 277/548] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:00.626902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335da42-95a7-4e86-828b-1d848c553f50 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:00.627248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335da42-95a7-4e86-828b-1d848c553f50 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:00.627537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335da42-95a7-4e86-828b-1d848c553f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:00.646216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3335da42-95a7-4e86-828b-1d848c553f50 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:00.647460 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3335da42-95a7-4e86-828b-1d848c553f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:00.679151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:00.696931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:00.730052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 278/549] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:24:00.766340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335da42-95a7-4e86-828b-1d848c553f50 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:00.767760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a6a33-628b-469f-a2fc-bbaed4301fb0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:00.829825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03aa77-b213-49c9-861a-aa921c118edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:00.830476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03aa77-b213-49c9-861a-aa921c118edc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:00.830760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03aa77-b213-49c9-861a-aa921c118edc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:00.831038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03aa77-b213-49c9-861a-aa921c118edc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:00.850282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03aa77-b213-49c9-861a-aa921c118edc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:00.851647 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03aa77-b213-49c9-861a-aa921c118edc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6e03aa77-b213-49c9-861a-aa921c118edc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:00.880023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:00.881135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 272/550] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Aug 4 07:24:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:00.882047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03aa77-b213-49c9-861a-aa921c118edc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:00.928429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335da42-95a7-4e86-828b-1d848c553f50 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:00.971385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922cf97-fc3a-4ab1-88b8-062bc7b67c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:00.972585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922cf97-fc3a-4ab1-88b8-062bc7b67c71 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:00.972882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922cf97-fc3a-4ab1-88b8-062bc7b67c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:00.973161 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922cf97-fc3a-4ab1-88b8-062bc7b67c71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.001222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922cf97-fc3a-4ab1-88b8-062bc7b67c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e922cf97-fc3a-4ab1-88b8-062bc7b67c71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:01.006696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:01.007774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:01.008620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 279/551] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:01.017179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922cf97-fc3a-4ab1-88b8-062bc7b67c71 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:01.048543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:01.070508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03aa77-b213-49c9-861a-aa921c118edc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:01.112200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:01.133843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03def309-4ee9-4323-b725-071625ea74d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:01.134464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03def309-4ee9-4323-b725-071625ea74d9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.134750 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03def309-4ee9-4323-b725-071625ea74d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.135027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03def309-4ee9-4323-b725-071625ea74d9 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.135828 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03def309-4ee9-4323-b725-071625ea74d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:01.136171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03def309-4ee9-4323-b725-071625ea74d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:01.137678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03def309-4ee9-4323-b725-071625ea74d9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:01.154542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922cf97-fc3a-4ab1-88b8-062bc7b67c71 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:01.156426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 273/552] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Aug 4 07:24:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:01.157663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a31e8-0dde-4d59-b6e3-d0bad73dcbe5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.157952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a31e8-0dde-4d59-b6e3-d0bad73dcbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.158235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a31e8-0dde-4d59-b6e3-d0bad73dcbe5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.164497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 274/553] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:01.165661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c48775-34d2-4713-bf26-6f315884d141 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.165956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c48775-34d2-4713-bf26-6f315884d141 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.166237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c48775-34d2-4713-bf26-6f315884d141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.184995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b8c48775-34d2-4713-bf26-6f315884d141 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:01.186189 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b8c48775-34d2-4713-bf26-6f315884d141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:01.202680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc74970-8da1-437d-9c11-c3a5359c82b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:01.203587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc74970-8da1-437d-9c11-c3a5359c82b0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.203873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc74970-8da1-437d-9c11-c3a5359c82b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.204181 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc74970-8da1-437d-9c11-c3a5359c82b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.205257 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc74970-8da1-437d-9c11-c3a5359c82b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:01.205449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc74970-8da1-437d-9c11-c3a5359c82b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:01.207799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc74970-8da1-437d-9c11-c3a5359c82b0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:01.224079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 280/554] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:01.225049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc74970-8da1-437d-9c11-c3a5359c82b0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:01.272203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 275/555] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:01.273268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c48775-34d2-4713-bf26-6f315884d141 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:01.342246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:01.355722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc74970-8da1-437d-9c11-c3a5359c82b0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:01.391645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c48775-34d2-4713-bf26-6f315884d141 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:01.392414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3da05d-b342-41b8-905c-3694ca99333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:01.393050 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3da05d-b342-41b8-905c-3694ca99333a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.393335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3da05d-b342-41b8-905c-3694ca99333a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.393612 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3da05d-b342-41b8-905c-3694ca99333a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.394468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:01.404548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3da05d-b342-41b8-905c-3694ca99333a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:01.406023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3da05d-b342-41b8-905c-3694ca99333a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3f3da05d-b342-41b8-905c-3694ca99333a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:01.421818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:01.422699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:01.428869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:01.428869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:01.428869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:01.428869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:01.428869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 281/556] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:01.428869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3da05d-b342-41b8-905c-3694ca99333a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:01.433886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3d813-a893-4413-8f0d-e3c40458b665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:01.434532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3d813-a893-4413-8f0d-e3c40458b665 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.434824 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3d813-a893-4413-8f0d-e3c40458b665 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.435106 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3d813-a893-4413-8f0d-e3c40458b665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.445586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3d813-a893-4413-8f0d-e3c40458b665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:01.447027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3d813-a893-4413-8f0d-e3c40458b665 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-bed3d813-a893-4413-8f0d-e3c40458b665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:01.461979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:01.462875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:01.464272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:01.464272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:01.464272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:01.464272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:01.464272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 276/557] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:01.464272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c48a3-1e84-42a7-8928-93181ecb0c94 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.464608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c48a3-1e84-42a7-8928-93181ecb0c94 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.465123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c48a3-1e84-42a7-8928-93181ecb0c94 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.470209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 277/558] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:01.470209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c48a3-1e84-42a7-8928-93181ecb0c94 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:01.540605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3da05d-b342-41b8-905c-3694ca99333a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:01.574010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c48a3-1e84-42a7-8928-93181ecb0c94 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:01.574526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767c50e-26dd-4906-9787-688d18b23be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:01.575188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767c50e-26dd-4906-9787-688d18b23be9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.575584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767c50e-26dd-4906-9787-688d18b23be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.575884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767c50e-26dd-4906-9787-688d18b23be9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.584111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:01.587913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767c50e-26dd-4906-9787-688d18b23be9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f767c50e-26dd-4906-9787-688d18b23be9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:01.592808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:01.595522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 282/559] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:01.596472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767c50e-26dd-4906-9787-688d18b23be9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:01.614755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f93bb-479d-447c-9359-bb137e17ec3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:01.615377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f93bb-479d-447c-9359-bb137e17ec3d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.615660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f93bb-479d-447c-9359-bb137e17ec3d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.615936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f93bb-479d-447c-9359-bb137e17ec3d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.616671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f93bb-479d-447c-9359-bb137e17ec3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:01.616978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f93bb-479d-447c-9359-bb137e17ec3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:01.618541 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f93bb-479d-447c-9359-bb137e17ec3d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:01.631939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 278/560] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:01.633018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e604-4a6c-41b7-959b-6daf9e80b74f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.633306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e604-4a6c-41b7-959b-6daf9e80b74f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.633584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e604-4a6c-41b7-959b-6daf9e80b74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.652918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8533e604-4a6c-41b7-959b-6daf9e80b74f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:01.654032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8533e604-4a6c-41b7-959b-6daf9e80b74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:01.665415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:01.708961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767c50e-26dd-4906-9787-688d18b23be9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:01.726082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 279/561] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:01.731189 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533e604-4a6c-41b7-959b-6daf9e80b74f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:01.747170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0baf8a-c53d-4a04-9789-8a1d7a34f453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:01.748226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0baf8a-c53d-4a04-9789-8a1d7a34f453 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.748515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0baf8a-c53d-4a04-9789-8a1d7a34f453 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.748794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0baf8a-c53d-4a04-9789-8a1d7a34f453 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.749895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0baf8a-c53d-4a04-9789-8a1d7a34f453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:01.750208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0baf8a-c53d-4a04-9789-8a1d7a34f453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:01.751904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0baf8a-c53d-4a04-9789-8a1d7a34f453 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:01.764478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 283/562] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:01.765576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0baf8a-c53d-4a04-9789-8a1d7a34f453 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:01.826276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:01.855518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533e604-4a6c-41b7-959b-6daf9e80b74f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:01.867358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0baf8a-c53d-4a04-9789-8a1d7a34f453 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:01.895184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fff196-7b9f-4bd6-9e37-17d1b9052777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:01.895747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff196-7b9f-4bd6-9e37-17d1b9052777 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.896233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff196-7b9f-4bd6-9e37-17d1b9052777 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.896412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff196-7b9f-4bd6-9e37-17d1b9052777 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.897161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fff196-7b9f-4bd6-9e37-17d1b9052777 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:01.897468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fff196-7b9f-4bd6-9e37-17d1b9052777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:01.898894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fff196-7b9f-4bd6-9e37-17d1b9052777 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:01.905861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bb4e7-592c-44e3-b8a3-e96588ecdb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:01.906704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bb4e7-592c-44e3-b8a3-e96588ecdb20 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:01.906822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 280/563] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:01.907257 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bb4e7-592c-44e3-b8a3-e96588ecdb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:01.907495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fff196-7b9f-4bd6-9e37-17d1b9052777 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:01.907865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bb4e7-592c-44e3-b8a3-e96588ecdb20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:01.919167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60bb4e7-592c-44e3-b8a3-e96588ecdb20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:01.921084 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60bb4e7-592c-44e3-b8a3-e96588ecdb20 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:01.922077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f60bb4e7-592c-44e3-b8a3-e96588ecdb20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:01.934091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:01.937274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:01.938168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:01.938168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:01.938168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:01.938168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:01.938168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:01.938168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:01.938168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:01.938168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:01.938168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 284/564] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:01.938168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bb4e7-592c-44e3-b8a3-e96588ecdb20 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:02.032634 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fff196-7b9f-4bd6-9e37-17d1b9052777 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:02.052144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bb4e7-592c-44e3-b8a3-e96588ecdb20 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:02.068732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f436fbd-ab9f-4e85-bb8c-882ea751e0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:02.069371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f436fbd-ab9f-4e85-bb8c-882ea751e0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.069659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f436fbd-ab9f-4e85-bb8c-882ea751e0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.069938 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f436fbd-ab9f-4e85-bb8c-882ea751e0f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.074526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:02.080375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f436fbd-ab9f-4e85-bb8c-882ea751e0f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3f436fbd-ab9f-4e85-bb8c-882ea751e0f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:02.086212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:02.087793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 281/565] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:02.088665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f436fbd-ab9f-4e85-bb8c-882ea751e0f6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:02.091912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6261d-4bb9-49df-94bb-b5dfc9892ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:02.092828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6261d-4bb9-49df-94bb-b5dfc9892ecb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.093119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6261d-4bb9-49df-94bb-b5dfc9892ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.093400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6261d-4bb9-49df-94bb-b5dfc9892ecb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.101682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6261d-4bb9-49df-94bb-b5dfc9892ecb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:02.103136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6261d-4bb9-49df-94bb-b5dfc9892ecb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-10d6261d-4bb9-49df-94bb-b5dfc9892ecb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:02.116601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:02.117554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 285/566] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:02.118482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6261d-4bb9-49df-94bb-b5dfc9892ecb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:02.166063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:02.200373 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f436fbd-ab9f-4e85-bb8c-882ea751e0f6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:02.227474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6261d-4bb9-49df-94bb-b5dfc9892ecb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:02.234140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fde8b-5824-4d9d-be8d-9f88d9116bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:02.234788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fde8b-5824-4d9d-be8d-9f88d9116bfc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.235055 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fde8b-5824-4d9d-be8d-9f88d9116bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.235332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fde8b-5824-4d9d-be8d-9f88d9116bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.246554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fde8b-5824-4d9d-be8d-9f88d9116bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:02.248034 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fde8b-5824-4d9d-be8d-9f88d9116bfc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:02.261865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c381791-7b5d-432a-bf73-b77da5276c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:02.262475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c381791-7b5d-432a-bf73-b77da5276c46 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.262761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c381791-7b5d-432a-bf73-b77da5276c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.263037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c381791-7b5d-432a-bf73-b77da5276c46 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1e8fde8b-5824-4d9d-be8d-9f88d9116bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:02.263443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:02.264514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:02.271217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:02.271217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:02.271217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:02.271217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:02.271489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c381791-7b5d-432a-bf73-b77da5276c46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:02.271489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c381791-7b5d-432a-bf73-b77da5276c46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:02.271651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 282/567] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:02.271734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c381791-7b5d-432a-bf73-b77da5276c46 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:02.271815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fde8b-5824-4d9d-be8d-9f88d9116bfc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:02.282063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 286/568] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:02.282876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c381791-7b5d-432a-bf73-b77da5276c46 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:02.325522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:02.368149 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fde8b-5824-4d9d-be8d-9f88d9116bfc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:02.385659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c381791-7b5d-432a-bf73-b77da5276c46 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:02.387308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:02.400779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b65ab-d893-4a50-b150-79bd5fbbbad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:02.401424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b65ab-d893-4a50-b150-79bd5fbbbad5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.401762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b65ab-d893-4a50-b150-79bd5fbbbad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.402040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b65ab-d893-4a50-b150-79bd5fbbbad5 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.402793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b65ab-d893-4a50-b150-79bd5fbbbad5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:02.403099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b65ab-d893-4a50-b150-79bd5fbbbad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:02.404521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b65ab-d893-4a50-b150-79bd5fbbbad5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:02.415048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 283/569] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:02.416323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9e668-a4a7-43a2-bb2a-e543fa02d3a5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.416549 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9e668-a4a7-43a2-bb2a-e543fa02d3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.416834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9e668-a4a7-43a2-bb2a-e543fa02d3a5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.418314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 284/570] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:02.419020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9e668-a4a7-43a2-bb2a-e543fa02d3a5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:02.423689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b20251-bb7f-45b3-8718-af4a90010fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:02.424340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b20251-bb7f-45b3-8718-af4a90010fdd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.424624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b20251-bb7f-45b3-8718-af4a90010fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.424900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b20251-bb7f-45b3-8718-af4a90010fdd None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.426060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b20251-bb7f-45b3-8718-af4a90010fdd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:02.426376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b20251-bb7f-45b3-8718-af4a90010fdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:02.427820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b20251-bb7f-45b3-8718-af4a90010fdd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:02.445249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 287/571] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Aug 4 07:24:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:02.446014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b20251-bb7f-45b3-8718-af4a90010fdd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:02.520212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9e668-a4a7-43a2-bb2a-e543fa02d3a5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:02.540926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:02.544571 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b20251-bb7f-45b3-8718-af4a90010fdd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:02.553911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba79e61-97a6-43f6-9c38-bebf61601596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:02.554530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba79e61-97a6-43f6-9c38-bebf61601596 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.554814 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba79e61-97a6-43f6-9c38-bebf61601596 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.555088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba79e61-97a6-43f6-9c38-bebf61601596 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.555923 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba79e61-97a6-43f6-9c38-bebf61601596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:02.556265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba79e61-97a6-43f6-9c38-bebf61601596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:02.557632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba79e61-97a6-43f6-9c38-bebf61601596 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:02.569072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 285/572] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:02.569902 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba79e61-97a6-43f6-9c38-bebf61601596 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:02.583461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc988df-47fa-4263-b8e1-06ab4c78077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:02.584158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc988df-47fa-4263-b8e1-06ab4c78077f None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.584446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc988df-47fa-4263-b8e1-06ab4c78077f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.584733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc988df-47fa-4263-b8e1-06ab4c78077f None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/4eedd711c6114de1bfcdf692ce058794/roles/88637a5a7c9d47aaacf803a09abc1755` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.599448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc988df-47fa-4263-b8e1-06ab4c78077f None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=4eedd711c6114de1bfcdf692ce058794, role_id=88637a5a7c9d47aaacf803a09abc1755)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:02.600937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc988df-47fa-4263-b8e1-06ab4c78077f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:02.611652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:02.666648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba79e61-97a6-43f6-9c38-bebf61601596 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:02.676823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 288/573] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:24:02 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/4eedd711c6114de1bfcdf692ce058794/roles/88637a5a7c9d47aaacf803a09abc1755 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:24:02.678195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe349a7-a22e-4f38-b10b-207e5677776b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.678537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe349a7-a22e-4f38-b10b-207e5677776b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.678788 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe349a7-a22e-4f38-b10b-207e5677776b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.698429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bbe349a7-a22e-4f38-b10b-207e5677776b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:02.699778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bbe349a7-a22e-4f38-b10b-207e5677776b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:02.703044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d52b4d-6e62-456b-960d-714cfa9dd845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:02.703044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d52b4d-6e62-456b-960d-714cfa9dd845 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.703273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d52b4d-6e62-456b-960d-714cfa9dd845 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.704463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d52b4d-6e62-456b-960d-714cfa9dd845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.718592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d52b4d-6e62-456b-960d-714cfa9dd845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:02.720132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d52b4d-6e62-456b-960d-714cfa9dd845 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-10d52b4d-6e62-456b-960d-714cfa9dd845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:02.738641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:02.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:02.740705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:02.740705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:02.740705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:02.740705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:02.740705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 286/574] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:02.741037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d52b4d-6e62-456b-960d-714cfa9dd845 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:02.788009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 289/575] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:02.789235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe349a7-a22e-4f38-b10b-207e5677776b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:02.852367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:02.861242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d52b4d-6e62-456b-960d-714cfa9dd845 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:02.890397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:02.898727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a84097-f2c8-4a56-a2b6-1ccac042dade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:02.899346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a84097-f2c8-4a56-a2b6-1ccac042dade None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.899623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a84097-f2c8-4a56-a2b6-1ccac042dade None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.899899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a84097-f2c8-4a56-a2b6-1ccac042dade None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.914676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe349a7-a22e-4f38-b10b-207e5677776b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:02.954697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267886ae-1e16-4de6-b5c4-0d5d8ef45e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:02.955340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267886ae-1e16-4de6-b5c4-0d5d8ef45e32 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:02.955621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267886ae-1e16-4de6-b5c4-0d5d8ef45e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:02.955893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267886ae-1e16-4de6-b5c4-0d5d8ef45e32 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:02.967205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267886ae-1e16-4de6-b5c4-0d5d8ef45e32 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:02.968693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267886ae-1e16-4de6-b5c4-0d5d8ef45e32 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:02.974337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a84097-f2c8-4a56-a2b6-1ccac042dade None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:02.976030 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a84097-f2c8-4a56-a2b6-1ccac042dade None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-267886ae-1e16-4de6-b5c4-0d5d8ef45e32 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:02.981506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:02.982663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:02.983877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:02.983877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:02.983877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:02.983877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:02.983877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:02.983877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:02.983877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:02.983877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:02.983877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 07:24:02.983877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:02.983877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 290/576] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:02.984500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267886ae-1e16-4de6-b5c4-0d5d8ef45e32 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.082754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:03.088140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267886ae-1e16-4de6-b5c4-0d5d8ef45e32 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:03.090122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 287/577] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:24:03.090924 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a84097-f2c8-4a56-a2b6-1ccac042dade None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.129015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7b73f-0170-4ecf-ae20-5ee90c20cc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:03.129914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7b73f-0170-4ecf-ae20-5ee90c20cc00 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:03.130344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7b73f-0170-4ecf-ae20-5ee90c20cc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:03.130780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7b73f-0170-4ecf-ae20-5ee90c20cc00 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:03.131627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7b73f-0170-4ecf-ae20-5ee90c20cc00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:03.132140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7b73f-0170-4ecf-ae20-5ee90c20cc00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:03.133511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7b73f-0170-4ecf-ae20-5ee90c20cc00 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:03.140725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:03.146829 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 291/578] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:03.147798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7b73f-0170-4ecf-ae20-5ee90c20cc00 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.202482 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a84097-f2c8-4a56-a2b6-1ccac042dade None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:03.241758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9451d33-de29-4d28-8bd4-34fe58c458e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:03.241758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9451d33-de29-4d28-8bd4-34fe58c458e0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:03.241758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9451d33-de29-4d28-8bd4-34fe58c458e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:03.241758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9451d33-de29-4d28-8bd4-34fe58c458e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:03.249806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9451d33-de29-4d28-8bd4-34fe58c458e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d9451d33-de29-4d28-8bd4-34fe58c458e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:03.254888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:03.256061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 288/579] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:03.257089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9451d33-de29-4d28-8bd4-34fe58c458e0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.299921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7b73f-0170-4ecf-ae20-5ee90c20cc00 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:03.315930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:03.333631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732dfb5-0f84-46cd-9bbc-633d67597184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:03.334240 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732dfb5-0f84-46cd-9bbc-633d67597184 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:03.334526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732dfb5-0f84-46cd-9bbc-633d67597184 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:03.334802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732dfb5-0f84-46cd-9bbc-633d67597184 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:03.335597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732dfb5-0f84-46cd-9bbc-633d67597184 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:03.335904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732dfb5-0f84-46cd-9bbc-633d67597184 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:03.337382 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732dfb5-0f84-46cd-9bbc-633d67597184 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:03.360902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 292/580] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Aug 4 07:24:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:03.360902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732dfb5-0f84-46cd-9bbc-633d67597184 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.369778 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9451d33-de29-4d28-8bd4-34fe58c458e0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:03.406122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555cc29-33d1-475c-b4f4-333b2043f0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:03.406757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555cc29-33d1-475c-b4f4-333b2043f0ef None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:03.407083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555cc29-33d1-475c-b4f4-333b2043f0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:03.407362 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555cc29-33d1-475c-b4f4-333b2043f0ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:03.415450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:03.420191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555cc29-33d1-475c-b4f4-333b2043f0ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:03.421639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555cc29-33d1-475c-b4f4-333b2043f0ef None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1555cc29-33d1-475c-b4f4-333b2043f0ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:03.437105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:03.438111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:03.438989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:03.438989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:03.438989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:03.438989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:03.438989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 289/581] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:03.439273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555cc29-33d1-475c-b4f4-333b2043f0ef None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.468499 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732dfb5-0f84-46cd-9bbc-633d67597184 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:03.503207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90c516-d40a-4104-bc64-7f063deac2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:03.504427 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90c516-d40a-4104-bc64-7f063deac2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:03.504711 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90c516-d40a-4104-bc64-7f063deac2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:03.505150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90c516-d40a-4104-bc64-7f063deac2a7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:03.512985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90c516-d40a-4104-bc64-7f063deac2a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:03.514475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90c516-d40a-4104-bc64-7f063deac2a7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-be90c516-d40a-4104-bc64-7f063deac2a7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:03.526229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:03.527168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 293/582] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:03.528606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90c516-d40a-4104-bc64-7f063deac2a7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.548414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555cc29-33d1-475c-b4f4-333b2043f0ef None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:03.555708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:03.582639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf16c06-b6fd-4d48-9032-c90b30a8df03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:03.583253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf16c06-b6fd-4d48-9032-c90b30a8df03 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:03.583534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf16c06-b6fd-4d48-9032-c90b30a8df03 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:03.583805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf16c06-b6fd-4d48-9032-c90b30a8df03 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:03.584547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf16c06-b6fd-4d48-9032-c90b30a8df03 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:03.584853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf16c06-b6fd-4d48-9032-c90b30a8df03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:03.586209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf16c06-b6fd-4d48-9032-c90b30a8df03 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:03.599239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 290/583] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:03.599239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf16c06-b6fd-4d48-9032-c90b30a8df03 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.634958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90c516-d40a-4104-bc64-7f063deac2a7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:03.651008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:03.668017 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d851c4-f392-415c-bccd-d1989a8c8a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:03.668661 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d851c4-f392-415c-bccd-d1989a8c8a0a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:03.668942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d851c4-f392-415c-bccd-d1989a8c8a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:03.669214 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d851c4-f392-415c-bccd-d1989a8c8a0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:03.680932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d851c4-f392-415c-bccd-d1989a8c8a0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:03.682808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d851c4-f392-415c-bccd-d1989a8c8a0a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-56d851c4-f392-415c-bccd-d1989a8c8a0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:03.699085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:03.700022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:03.701093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:03.701093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:03.701093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:03.701093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:03.704635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 294/584] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:03.705452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d851c4-f392-415c-bccd-d1989a8c8a0a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.710464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf16c06-b6fd-4d48-9032-c90b30a8df03 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:03.747413 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7b3d4-d517-4ae6-8003-de7fcc3a0729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:03.748343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7b3d4-d517-4ae6-8003-de7fcc3a0729 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:03.748797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7b3d4-d517-4ae6-8003-de7fcc3a0729 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:03.749212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7b3d4-d517-4ae6-8003-de7fcc3a0729 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:03.750162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7b3d4-d517-4ae6-8003-de7fcc3a0729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:03.750641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7b3d4-d517-4ae6-8003-de7fcc3a0729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:03.752260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7b3d4-d517-4ae6-8003-de7fcc3a0729 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:03.764947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 291/585] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:03.765777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7b3d4-d517-4ae6-8003-de7fcc3a0729 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.796164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:03.813940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d851c4-f392-415c-bccd-d1989a8c8a0a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:03.850040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a19f1-98de-46e0-8f5e-5d7700bbe095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:03.850663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a19f1-98de-46e0-8f5e-5d7700bbe095 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:03.850941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a19f1-98de-46e0-8f5e-5d7700bbe095 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:03.851214 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a19f1-98de-46e0-8f5e-5d7700bbe095 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:03.852016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a19f1-98de-46e0-8f5e-5d7700bbe095 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:03.852325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a19f1-98de-46e0-8f5e-5d7700bbe095 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:03.853733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a19f1-98de-46e0-8f5e-5d7700bbe095 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:03.875833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 295/586] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Aug 4 07:24:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:03.875833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a19f1-98de-46e0-8f5e-5d7700bbe095 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.881593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7b3d4-d517-4ae6-8003-de7fcc3a0729 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:03.899417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:03.919719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b4393-2d0a-4544-a926-6c4e373f1b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:03.920380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b4393-2d0a-4544-a926-6c4e373f1b86 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:03.920660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b4393-2d0a-4544-a926-6c4e373f1b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:03.920931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b4393-2d0a-4544-a926-6c4e373f1b86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:03.932442 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b4393-2d0a-4544-a926-6c4e373f1b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:03.934068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b4393-2d0a-4544-a926-6c4e373f1b86 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-753b4393-2d0a-4544-a926-6c4e373f1b86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:03.950747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:03.951849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:03.952985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:03.952985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:03.952985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:03.952985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:03.958796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 292/587] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:03.958796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b4393-2d0a-4544-a926-6c4e373f1b86 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:03.985267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a19f1-98de-46e0-8f5e-5d7700bbe095 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.019710 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa6c7d-a564-4bd1-8288-4022ed114083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.020413 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa6c7d-a564-4bd1-8288-4022ed114083 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.020788 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa6c7d-a564-4bd1-8288-4022ed114083 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.021220 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa6c7d-a564-4bd1-8288-4022ed114083 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.021928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa6c7d-a564-4bd1-8288-4022ed114083 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:04.022332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa6c7d-a564-4bd1-8288-4022ed114083 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.023805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa6c7d-a564-4bd1-8288-4022ed114083 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:04.035663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 296/588] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:04.040729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa6c7d-a564-4bd1-8288-4022ed114083 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:04.043025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:04.062616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b4393-2d0a-4544-a926-6c4e373f1b86 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.098635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcad6d-acc9-433e-9f91-330ee79ed0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.099265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcad6d-acc9-433e-9f91-330ee79ed0bc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.099553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcad6d-acc9-433e-9f91-330ee79ed0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.099831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcad6d-acc9-433e-9f91-330ee79ed0bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.106823 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcad6d-acc9-433e-9f91-330ee79ed0bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.108556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcad6d-acc9-433e-9f91-330ee79ed0bc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d2fcad6d-acc9-433e-9f91-330ee79ed0bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:04.120661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:04.121556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:04.122522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:04.122522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:04.122522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:04.122522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:04.122522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:04.122522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:04.122522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:04.122522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:04.122522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 293/589] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:04.123576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcad6d-acc9-433e-9f91-330ee79ed0bc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:04.132990 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:04.148377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa6c7d-a564-4bd1-8288-4022ed114083 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.189798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4906ab-10d7-47a6-b8a4-242658d57f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.190939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4906ab-10d7-47a6-b8a4-242658d57f7e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.191372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4906ab-10d7-47a6-b8a4-242658d57f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.191795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4906ab-10d7-47a6-b8a4-242658d57f7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.202795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4906ab-10d7-47a6-b8a4-242658d57f7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.204390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4906ab-10d7-47a6-b8a4-242658d57f7e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cd4906ab-10d7-47a6-b8a4-242658d57f7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:04.219865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:04.220964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:04.221860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:04.221860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:04.221860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:04.221860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:04.221860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 297/590] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:04.221860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4906ab-10d7-47a6-b8a4-242658d57f7e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:04.239879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcad6d-acc9-433e-9f91-330ee79ed0bc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.277500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4b60b-039e-48fd-b15a-0d54d30b5430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.278440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4b60b-039e-48fd-b15a-0d54d30b5430 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.278890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4b60b-039e-48fd-b15a-0d54d30b5430 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.279327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4b60b-039e-48fd-b15a-0d54d30b5430 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.280056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4b60b-039e-48fd-b15a-0d54d30b5430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:04.280367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4b60b-039e-48fd-b15a-0d54d30b5430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.281951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4b60b-039e-48fd-b15a-0d54d30b5430 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:04.293373 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:04.302492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 294/591] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:04.302492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4b60b-039e-48fd-b15a-0d54d30b5430 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:04.330783 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4906ab-10d7-47a6-b8a4-242658d57f7e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.372867 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb400d-b8e5-4ba5-b1eb-cc8febbd9691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.373522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb400d-b8e5-4ba5-b1eb-cc8febbd9691 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.373793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb400d-b8e5-4ba5-b1eb-cc8febbd9691 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.374057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb400d-b8e5-4ba5-b1eb-cc8febbd9691 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.380725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb400d-b8e5-4ba5-b1eb-cc8febbd9691 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.382177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb400d-b8e5-4ba5-b1eb-cc8febbd9691 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:04.383903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fadb400d-b8e5-4ba5-b1eb-cc8febbd9691 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:04.394722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:04.395916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:04.396897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:04.396897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:04.396897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:04.396897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:04.396897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:04.396897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:04.396897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:04.396897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:04.396897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 298/592] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:04.397381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb400d-b8e5-4ba5-b1eb-cc8febbd9691 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:04.420473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4b60b-039e-48fd-b15a-0d54d30b5430 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.455857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6df359-d5d0-4f52-a17a-b06da04afd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.457138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6df359-d5d0-4f52-a17a-b06da04afd1d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.457458 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6df359-d5d0-4f52-a17a-b06da04afd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.457677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6df359-d5d0-4f52-a17a-b06da04afd1d None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.458423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6df359-d5d0-4f52-a17a-b06da04afd1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:04.458961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6df359-d5d0-4f52-a17a-b06da04afd1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.460182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6df359-d5d0-4f52-a17a-b06da04afd1d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:04.472598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 295/593] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:04.473395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6df359-d5d0-4f52-a17a-b06da04afd1d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:04.510358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb400d-b8e5-4ba5-b1eb-cc8febbd9691 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.537408 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:04.548773 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fe0e7-4f31-47ba-9374-fa86056ac57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.553104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fe0e7-4f31-47ba-9374-fa86056ac57a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.553104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fe0e7-4f31-47ba-9374-fa86056ac57a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.553104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fe0e7-4f31-47ba-9374-fa86056ac57a None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.553104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fe0e7-4f31-47ba-9374-fa86056ac57a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:04.553104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fe0e7-4f31-47ba-9374-fa86056ac57a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.553104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fe0e7-4f31-47ba-9374-fa86056ac57a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:04.570006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 299/594] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:04.570006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fe0e7-4f31-47ba-9374-fa86056ac57a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:04.584262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6df359-d5d0-4f52-a17a-b06da04afd1d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.619682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b103-d701-4b63-bcc1-0e99eb734bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.620385 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b103-d701-4b63-bcc1-0e99eb734bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.620667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b103-d701-4b63-bcc1-0e99eb734bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.620945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b103-d701-4b63-bcc1-0e99eb734bd8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.629116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5b103-d701-4b63-bcc1-0e99eb734bd8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.630548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5b103-d701-4b63-bcc1-0e99eb734bd8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:04.635599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-20f5b103-d701-4b63-bcc1-0e99eb734bd8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:04.644962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:04.652226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 296/595] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:04.653185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b103-d701-4b63-bcc1-0e99eb734bd8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:04.687151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fe0e7-4f31-47ba-9374-fa86056ac57a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.721733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac6212-cf63-42be-bd86-38eb2a0e8d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.722984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac6212-cf63-42be-bd86-38eb2a0e8d59 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.723263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac6212-cf63-42be-bd86-38eb2a0e8d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.723534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac6212-cf63-42be-bd86-38eb2a0e8d59 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.724308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac6212-cf63-42be-bd86-38eb2a0e8d59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:04.724610 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac6212-cf63-42be-bd86-38eb2a0e8d59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.726041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac6212-cf63-42be-bd86-38eb2a0e8d59 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:04.739079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 300/596] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:04.739883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac6212-cf63-42be-bd86-38eb2a0e8d59 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:04.761418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b103-d701-4b63-bcc1-0e99eb734bd8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.780201 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:04.794608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffde4f-470e-46d6-80b4-19b9af47c29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.795441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffde4f-470e-46d6-80b4-19b9af47c29b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.795727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffde4f-470e-46d6-80b4-19b9af47c29b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.796036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffde4f-470e-46d6-80b4-19b9af47c29b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.796829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffde4f-470e-46d6-80b4-19b9af47c29b None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=9503e2888d5e4831afc0d9d9522c3eea) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:04.797141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffde4f-470e-46d6-80b4-19b9af47c29b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.798563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffde4f-470e-46d6-80b4-19b9af47c29b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:04.804643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 297/597] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:04 2022] GET /identity/v3/role_assignments?user.id=9503e2888d5e4831afc0d9d9522c3eea&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:04.805409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffde4f-470e-46d6-80b4-19b9af47c29b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:04.843479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac6212-cf63-42be-bd86-38eb2a0e8d59 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.865392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:04.883273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b6b9cf-ef8d-43b5-b910-d75578bd06dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.884081 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6b9cf-ef8d-43b5-b910-d75578bd06dd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.884406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6b9cf-ef8d-43b5-b910-d75578bd06dd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.884720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6b9cf-ef8d-43b5-b910-d75578bd06dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.892396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b6b9cf-ef8d-43b5-b910-d75578bd06dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.893931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b6b9cf-ef8d-43b5-b910-d75578bd06dd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f2b6b9cf-ef8d-43b5-b910-d75578bd06dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:04.906579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:04.907717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:04.914907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:04.914907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:04.914907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:04.914907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:04.914907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:04.914907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:04.914907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:04.914907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:04.914907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 301/598] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:04.914907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7606c-6158-4cea-b010-7a25c54602b0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.914907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7606c-6158-4cea-b010-7a25c54602b0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.915655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7606c-6158-4cea-b010-7a25c54602b0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.915655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 302/599] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:04.915655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7606c-6158-4cea-b010-7a25c54602b0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:04.933998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffde4f-470e-46d6-80b4-19b9af47c29b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:04.979975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a55142-f023-4d57-9c1f-c99705fcfecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:04.980621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a55142-f023-4d57-9c1f-c99705fcfecb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:04.980905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a55142-f023-4d57-9c1f-c99705fcfecb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:04.981183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a55142-f023-4d57-9c1f-c99705fcfecb None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:04.981988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a55142-f023-4d57-9c1f-c99705fcfecb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:04.982309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a55142-f023-4d57-9c1f-c99705fcfecb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:04.984005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a55142-f023-4d57-9c1f-c99705fcfecb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:05.001570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 298/600] 10.222.0.64 () {56 vars in 1139 bytes} [Thu Aug 4 07:24:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:05.002794 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275b731-6493-454b-ba90-6700fdffcf42 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:05.003109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275b731-6493-454b-ba90-6700fdffcf42 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:05.004109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275b731-6493-454b-ba90-6700fdffcf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:05.021702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d275b731-6493-454b-ba90-6700fdffcf42 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:05.022897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d275b731-6493-454b-ba90-6700fdffcf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:05.044889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7606c-6158-4cea-b010-7a25c54602b0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:05.059469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:05.088516 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12469b4c-8434-4fd8-80cd-eb87ac52cb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:05.089497 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12469b4c-8434-4fd8-80cd-eb87ac52cb25 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:05.089951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12469b4c-8434-4fd8-80cd-eb87ac52cb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:05.090414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12469b4c-8434-4fd8-80cd-eb87ac52cb25 None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/eab096d4ccae4bc09e6177afd85a55de/roles/88637a5a7c9d47aaacf803a09abc1755` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:05.091536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 299/601] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:05.092390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275b731-6493-454b-ba90-6700fdffcf42 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:05.120268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12469b4c-8434-4fd8-80cd-eb87ac52cb25 None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=eab096d4ccae4bc09e6177afd85a55de, role_id=88637a5a7c9d47aaacf803a09abc1755)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:05.121736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12469b4c-8434-4fd8-80cd-eb87ac52cb25 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:05.190176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 303/602] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:24:04 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/eab096d4ccae4bc09e6177afd85a55de/roles/88637a5a7c9d47aaacf803a09abc1755 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:24:05.196464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275b731-6493-454b-ba90-6700fdffcf42 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:05.201071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12469b4c-8434-4fd8-80cd-eb87ac52cb25 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:05.246531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572dea0-3811-457a-95d9-b63a4a26d6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:05.247715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572dea0-3811-457a-95d9-b63a4a26d6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:05.248217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572dea0-3811-457a-95d9-b63a4a26d6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:05.248393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:05.248807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572dea0-3811-457a-95d9-b63a4a26d6f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:05.250270 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572dea0-3811-457a-95d9-b63a4a26d6f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:05.251096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572dea0-3811-457a-95d9-b63a4a26d6f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:05.253287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572dea0-3811-457a-95d9-b63a4a26d6f5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:05.278873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 300/603] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:05.278873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572dea0-3811-457a-95d9-b63a4a26d6f5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:05.323681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:05.361782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12469b4c-8434-4fd8-80cd-eb87ac52cb25 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:05.410358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64680976-9758-414b-a2e5-6aebd5076ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:05.410966 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680976-9758-414b-a2e5-6aebd5076ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:05.411247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680976-9758-414b-a2e5-6aebd5076ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:05.411522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680976-9758-414b-a2e5-6aebd5076ed1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:05.415589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572dea0-3811-457a-95d9-b63a4a26d6f5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:05.422827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64680976-9758-414b-a2e5-6aebd5076ed1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:05.424337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64680976-9758-414b-a2e5-6aebd5076ed1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-64680976-9758-414b-a2e5-6aebd5076ed1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:05.439929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:05.441599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:05.442500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:05.442500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:05.442500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:05.442500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:05.442500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 304/604] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:05.442781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64680976-9758-414b-a2e5-6aebd5076ed1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:05.463126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb5f16-efd1-4127-8b25-7cea953f181b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:05.464070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb5f16-efd1-4127-8b25-7cea953f181b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:05.464496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb5f16-efd1-4127-8b25-7cea953f181b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:05.464915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb5f16-efd1-4127-8b25-7cea953f181b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:05.483949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb5f16-efd1-4127-8b25-7cea953f181b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b3cb5f16-efd1-4127-8b25-7cea953f181b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:05.491182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:05.492391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:05.493418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:05.496438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 301/605] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:05.497615 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb5f16-efd1-4127-8b25-7cea953f181b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:05.541027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:05.560341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64680976-9758-414b-a2e5-6aebd5076ed1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:05.597446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddad57-c3a9-45aa-8201-159e2c5cb98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:05.598200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddad57-c3a9-45aa-8201-159e2c5cb98d None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:05.598584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddad57-c3a9-45aa-8201-159e2c5cb98d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:05.599583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddad57-c3a9-45aa-8201-159e2c5cb98d None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/56dfc4dfda424d40b7c9f17345b8ad45/roles/88637a5a7c9d47aaacf803a09abc1755` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:05.614627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddad57-c3a9-45aa-8201-159e2c5cb98d None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=56dfc4dfda424d40b7c9f17345b8ad45, role_id=88637a5a7c9d47aaacf803a09abc1755)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:05.616226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddad57-c3a9-45aa-8201-159e2c5cb98d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:05.626619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:05.636065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb5f16-efd1-4127-8b25-7cea953f181b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:05.669464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d33147-067f-49a7-a768-2bf039241892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:05.670075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33147-067f-49a7-a768-2bf039241892 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:05.670352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33147-067f-49a7-a768-2bf039241892 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:05.670621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33147-067f-49a7-a768-2bf039241892 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:05.671308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d33147-067f-49a7-a768-2bf039241892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:05.671610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d33147-067f-49a7-a768-2bf039241892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:05.673002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d33147-067f-49a7-a768-2bf039241892 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:05.684825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 302/606] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:05.685513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d33147-067f-49a7-a768-2bf039241892 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:05.713705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 305/607] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:24:05 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/56dfc4dfda424d40b7c9f17345b8ad45/roles/88637a5a7c9d47aaacf803a09abc1755 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:24:05.714569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddad57-c3a9-45aa-8201-159e2c5cb98d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:05.845782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddad57-c3a9-45aa-8201-159e2c5cb98d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:05.876263 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d33147-067f-49a7-a768-2bf039241892 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:05.885111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:05.891933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90853ac9-a452-4d5c-a66d-98c325447918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:05.892612 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90853ac9-a452-4d5c-a66d-98c325447918 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:05.892893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90853ac9-a452-4d5c-a66d-98c325447918 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:05.893159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90853ac9-a452-4d5c-a66d-98c325447918 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:05.901830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90853ac9-a452-4d5c-a66d-98c325447918 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:05.903567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90853ac9-a452-4d5c-a66d-98c325447918 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-90853ac9-a452-4d5c-a66d-98c325447918 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:05.920970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:05.922005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:05.922882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:05.922882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:05.922882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:05.922882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:05.922882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:05.922882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:05.922882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:05.922882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:05.922882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 306/608] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:05.929306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90853ac9-a452-4d5c-a66d-98c325447918 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:05.974479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca33d0-1344-4456-9bc7-80f67ce4d3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:05.975228 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca33d0-1344-4456-9bc7-80f67ce4d3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:05.975584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca33d0-1344-4456-9bc7-80f67ce4d3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:05.975865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca33d0-1344-4456-9bc7-80f67ce4d3a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:05.976625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca33d0-1344-4456-9bc7-80f67ce4d3a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:05.976953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca33d0-1344-4456-9bc7-80f67ce4d3a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:05.978305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca33d0-1344-4456-9bc7-80f67ce4d3a7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:05.998491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 303/609] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:06.003626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca33d0-1344-4456-9bc7-80f67ce4d3a7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:06.030319 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:06.066501 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90853ac9-a452-4d5c-a66d-98c325447918 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:06.102631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5f290-8130-4534-a49b-4ebd2e8edccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:06.103434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5f290-8130-4534-a49b-4ebd2e8edccc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:06.103809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5f290-8130-4534-a49b-4ebd2e8edccc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:06.104249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5f290-8130-4534-a49b-4ebd2e8edccc None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:06.105153 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5f290-8130-4534-a49b-4ebd2e8edccc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:06.105558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5f290-8130-4534-a49b-4ebd2e8edccc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:06.107666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5f290-8130-4534-a49b-4ebd2e8edccc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:06.120010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 307/610] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:06.129446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5f290-8130-4534-a49b-4ebd2e8edccc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:06.158589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca33d0-1344-4456-9bc7-80f67ce4d3a7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:06.213470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:06.237641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee043d5a-ebda-41f0-84a4-cd38205ba34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:06.238405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee043d5a-ebda-41f0-84a4-cd38205ba34f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:06.238693 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee043d5a-ebda-41f0-84a4-cd38205ba34f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:06.238973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee043d5a-ebda-41f0-84a4-cd38205ba34f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:06.260005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee043d5a-ebda-41f0-84a4-cd38205ba34f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:06.260005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee043d5a-ebda-41f0-84a4-cd38205ba34f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ee043d5a-ebda-41f0-84a4-cd38205ba34f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:06.275157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:06.276284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:06.277381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:06.277381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:06.277381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:06.277381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:06.282878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 304/611] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:06.283423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5f290-8130-4534-a49b-4ebd2e8edccc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:06.285157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee043d5a-ebda-41f0-84a4-cd38205ba34f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:06.356266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:06.370048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cef9c-d93c-424d-8830-7534f240da09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:06.370734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cef9c-d93c-424d-8830-7534f240da09 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:06.371023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cef9c-d93c-424d-8830-7534f240da09 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:06.371302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cef9c-d93c-424d-8830-7534f240da09 None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/b80e701a81cb4fa893e55288559010f5/roles/88637a5a7c9d47aaacf803a09abc1755` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:06.395112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee043d5a-ebda-41f0-84a4-cd38205ba34f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:06.401373 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cef9c-d93c-424d-8830-7534f240da09 None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=b80e701a81cb4fa893e55288559010f5, role_id=88637a5a7c9d47aaacf803a09abc1755)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:06.402846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cef9c-d93c-424d-8830-7534f240da09 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:06.432544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f59f2e-593e-4a97-ba2a-836075918abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:06.433204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f59f2e-593e-4a97-ba2a-836075918abf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:06.433487 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f59f2e-593e-4a97-ba2a-836075918abf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:06.433759 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f59f2e-593e-4a97-ba2a-836075918abf None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:06.434452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f59f2e-593e-4a97-ba2a-836075918abf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:06.434844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f59f2e-593e-4a97-ba2a-836075918abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:06.436368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f59f2e-593e-4a97-ba2a-836075918abf None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:06.449625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 305/612] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:06.454889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f59f2e-593e-4a97-ba2a-836075918abf None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:06.482492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 308/613] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:24:06 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/b80e701a81cb4fa893e55288559010f5/roles/88637a5a7c9d47aaacf803a09abc1755 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:24:06.613052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f59f2e-593e-4a97-ba2a-836075918abf None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:06.646687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:06.660948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dfd61-dd12-44b3-b041-9bb65e6c5510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:06.661605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dfd61-dd12-44b3-b041-9bb65e6c5510 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:06.667565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dfd61-dd12-44b3-b041-9bb65e6c5510 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:06.667565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dfd61-dd12-44b3-b041-9bb65e6c5510 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:06.670921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dfd61-dd12-44b3-b041-9bb65e6c5510 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:06.672414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dfd61-dd12-44b3-b041-9bb65e6c5510 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-461dfd61-dd12-44b3-b041-9bb65e6c5510 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:06.687603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:06.688662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:06.689597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:06.689597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:06.689597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:06.689597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:06.689597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:06.689597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:06.689597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:06.689597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:06.689597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:06.689597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:06.693381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 306/614] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Aug 4 07:24:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:06.699373 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cef9c-d93c-424d-8830-7534f240da09 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:06.816852 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:06.822686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cef9c-d93c-424d-8830-7534f240da09 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:06.982562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8a10d-f353-4f1c-b902-aac423d3a5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:06.983445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8a10d-f353-4f1c-b902-aac423d3a5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:06.983730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8a10d-f353-4f1c-b902-aac423d3a5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:06.984045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8a10d-f353-4f1c-b902-aac423d3a5e1 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:06.985066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8a10d-f353-4f1c-b902-aac423d3a5e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:06.985586 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8a10d-f353-4f1c-b902-aac423d3a5e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:06.987384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8a10d-f353-4f1c-b902-aac423d3a5e1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:07.006680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 309/615] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Aug 4 07:24:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:07.012191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dfd61-dd12-44b3-b041-9bb65e6c5510 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:07.145569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dfd61-dd12-44b3-b041-9bb65e6c5510 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:07.158455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dd623-b15e-4c9d-996c-d73ffb38f3ea None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:07.158764 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dd623-b15e-4c9d-996c-d73ffb38f3ea None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:07.159019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dd623-b15e-4c9d-996c-d73ffb38f3ea None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:07.161117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 310/616] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:07.167301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa87bc6-68f1-42ec-b82a-7451219121db None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:07.167585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa87bc6-68f1-42ec-b82a-7451219121db None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:07.167874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa87bc6-68f1-42ec-b82a-7451219121db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:07.200751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e9bd6-9189-460a-b5ad-8dc4dd34c23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:07.201776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e9bd6-9189-460a-b5ad-8dc4dd34c23d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:07.202286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e9bd6-9189-460a-b5ad-8dc4dd34c23d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:07.202834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e9bd6-9189-460a-b5ad-8dc4dd34c23d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:07.204431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-afa87bc6-68f1-42ec-b82a-7451219121db None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:07.208488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-afa87bc6-68f1-42ec-b82a-7451219121db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:07.289687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:07.305112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e9bd6-9189-460a-b5ad-8dc4dd34c23d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:07.306874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e9bd6-9189-460a-b5ad-8dc4dd34c23d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:07.341636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 311/617] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:07.402886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa87bc6-68f1-42ec-b82a-7451219121db None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:07.479664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 307/618] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:24:07.486202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e9bd6-9189-460a-b5ad-8dc4dd34c23d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:07.506952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:07.522671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa87bc6-68f1-42ec-b82a-7451219121db None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:07.583600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6e2dc-7b6a-4482-9bba-cb5ff41e14e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:07.584618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6e2dc-7b6a-4482-9bba-cb5ff41e14e8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:07.585107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6e2dc-7b6a-4482-9bba-cb5ff41e14e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:07.587154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6e2dc-7b6a-4482-9bba-cb5ff41e14e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:07.606911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6e2dc-7b6a-4482-9bba-cb5ff41e14e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1cb6e2dc-7b6a-4482-9bba-cb5ff41e14e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:07.613337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:07.614457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:07.615448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:07.619074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 312/619] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:07.625564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a851967-f289-4258-8b54-419c18c7359f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:07.625797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a851967-f289-4258-8b54-419c18c7359f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:07.626027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a851967-f289-4258-8b54-419c18c7359f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:07.627442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 313/620] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:07.629180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a851967-f289-4258-8b54-419c18c7359f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:07.649349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:07.667776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e9bd6-9189-460a-b5ad-8dc4dd34c23d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:07.710758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e7860-355b-4eda-9dc1-6cb1082a9c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:07.711716 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e7860-355b-4eda-9dc1-6cb1082a9c65 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:07.712076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e7860-355b-4eda-9dc1-6cb1082a9c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:07.712368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e7860-355b-4eda-9dc1-6cb1082a9c65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:07.726066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e7860-355b-4eda-9dc1-6cb1082a9c65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:07.727510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e7860-355b-4eda-9dc1-6cb1082a9c65 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6e9e7860-355b-4eda-9dc1-6cb1082a9c65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:07.747146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:07.748134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:07.749064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:07.749064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:07.749064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:07.749064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:07.752059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 308/621] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:07.754349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f081-fff7-4f0c-b110-c67b00c8598d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:07.754639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f081-fff7-4f0c-b110-c67b00c8598d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:07.754920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f081-fff7-4f0c-b110-c67b00c8598d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:07.765009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a851967-f289-4258-8b54-419c18c7359f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:07.774633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6289f081-fff7-4f0c-b110-c67b00c8598d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:07.775977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6289f081-fff7-4f0c-b110-c67b00c8598d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:07.801346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e37b55-8f06-4e73-a99b-e81ffc7ee302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:07.802325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e37b55-8f06-4e73-a99b-e81ffc7ee302 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:07.802810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e37b55-8f06-4e73-a99b-e81ffc7ee302 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:07.803308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e37b55-8f06-4e73-a99b-e81ffc7ee302 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:07.804268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e37b55-8f06-4e73-a99b-e81ffc7ee302 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:07.804778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e37b55-8f06-4e73-a99b-e81ffc7ee302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:07.806346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e37b55-8f06-4e73-a99b-e81ffc7ee302 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:07.815206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 314/622] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:07.816994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e37b55-8f06-4e73-a99b-e81ffc7ee302 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:07.829890 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:07.864768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 309/623] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:07.865883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289f081-fff7-4f0c-b110-c67b00c8598d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:07.931790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e37b55-8f06-4e73-a99b-e81ffc7ee302 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:07.952412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:07.969586 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab530a-d446-42a8-a5f2-068385fe4c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:07.969964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab530a-d446-42a8-a5f2-068385fe4c2d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:07.970307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab530a-d446-42a8-a5f2-068385fe4c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:07.970934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab530a-d446-42a8-a5f2-068385fe4c2d None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:07.971770 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab530a-d446-42a8-a5f2-068385fe4c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:07.972341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab530a-d446-42a8-a5f2-068385fe4c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:07.973831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab530a-d446-42a8-a5f2-068385fe4c2d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:07.991802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 315/624] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:07.992747 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab530a-d446-42a8-a5f2-068385fe4c2d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:07.999245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289f081-fff7-4f0c-b110-c67b00c8598d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:08.038621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6f5f9-f085-4796-9fc1-07bdbcbc54c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:08.039827 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6f5f9-f085-4796-9fc1-07bdbcbc54c7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.040311 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6f5f9-f085-4796-9fc1-07bdbcbc54c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.041241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6f5f9-f085-4796-9fc1-07bdbcbc54c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.054779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6f5f9-f085-4796-9fc1-07bdbcbc54c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:08.056401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6f5f9-f085-4796-9fc1-07bdbcbc54c7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:08.070079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8ef6f5f9-f085-4796-9fc1-07bdbcbc54c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:08.073063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:08.074155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:08.075280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:08.075280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:08.075280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:08.075280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:08.080014 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 310/625] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:08.080014 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6f5f9-f085-4796-9fc1-07bdbcbc54c7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:08.121058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab530a-d446-42a8-a5f2-068385fe4c2d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:08.173747 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03328d02-65fd-4888-8c63-52a95c1a666d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:08.174487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328d02-65fd-4888-8c63-52a95c1a666d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.174803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328d02-65fd-4888-8c63-52a95c1a666d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.175725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328d02-65fd-4888-8c63-52a95c1a666d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.185803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03328d02-65fd-4888-8c63-52a95c1a666d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-03328d02-65fd-4888-8c63-52a95c1a666d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:08.192087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:08.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:08.193854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 316/626] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:08.194562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03328d02-65fd-4888-8c63-52a95c1a666d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:08.201087 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6f5f9-f085-4796-9fc1-07bdbcbc54c7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:08.224117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:08.240868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080dac9-59c1-4b01-a749-2734001f85fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:08.241671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080dac9-59c1-4b01-a749-2734001f85fc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.242329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080dac9-59c1-4b01-a749-2734001f85fc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.242612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080dac9-59c1-4b01-a749-2734001f85fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.250506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080dac9-59c1-4b01-a749-2734001f85fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:08.252038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080dac9-59c1-4b01-a749-2734001f85fc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b080dac9-59c1-4b01-a749-2734001f85fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:08.264346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:08.265427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:08.266515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:08.266515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:08.266515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:08.266515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:08.266515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:08.266515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:08.266515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:08.266515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:08.274244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 311/627] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:08.275677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d7f42-222b-4bdc-85f1-61b6a43a7677 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.276351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d7f42-222b-4bdc-85f1-61b6a43a7677 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.276633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d7f42-222b-4bdc-85f1-61b6a43a7677 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.278586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 312/628] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:08.281228 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d7f42-222b-4bdc-85f1-61b6a43a7677 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:08.299634 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03328d02-65fd-4888-8c63-52a95c1a666d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:08.329155 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:08.335377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b527fce-f7ac-4303-b70a-c508120c7b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:08.340857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b527fce-f7ac-4303-b70a-c508120c7b91 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.340857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b527fce-f7ac-4303-b70a-c508120c7b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.340857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b527fce-f7ac-4303-b70a-c508120c7b91 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.340857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b527fce-f7ac-4303-b70a-c508120c7b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:08.340857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b527fce-f7ac-4303-b70a-c508120c7b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:08.340857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b527fce-f7ac-4303-b70a-c508120c7b91 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:08.352801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 317/629] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:08.354013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b527fce-f7ac-4303-b70a-c508120c7b91 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:08.402430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d7f42-222b-4bdc-85f1-61b6a43a7677 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:08.443859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b045ee-7fb0-4b92-aa06-a14f52bb9f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:08.444518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b045ee-7fb0-4b92-aa06-a14f52bb9f1b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.444816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b045ee-7fb0-4b92-aa06-a14f52bb9f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.445095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b045ee-7fb0-4b92-aa06-a14f52bb9f1b None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.445812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b045ee-7fb0-4b92-aa06-a14f52bb9f1b None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:08.446117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b045ee-7fb0-4b92-aa06-a14f52bb9f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:08.447484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b045ee-7fb0-4b92-aa06-a14f52bb9f1b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:08.460632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 313/630] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:08.460632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb384ee-4ff0-433a-b747-d4b64ef0be0a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.460632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb384ee-4ff0-433a-b747-d4b64ef0be0a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.460632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb384ee-4ff0-433a-b747-d4b64ef0be0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.465266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b527fce-f7ac-4303-b70a-c508120c7b91 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:08.477054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ddb384ee-4ff0-433a-b747-d4b64ef0be0a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:08.478480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ddb384ee-4ff0-433a-b747-d4b64ef0be0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:08.503214 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5c00c-c0cf-4c3c-8120-7df4bb09288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:08.503831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5c00c-c0cf-4c3c-8120-7df4bb09288b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.504148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5c00c-c0cf-4c3c-8120-7df4bb09288b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.504432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5c00c-c0cf-4c3c-8120-7df4bb09288b None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.505180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5c00c-c0cf-4c3c-8120-7df4bb09288b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:08.505490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5c00c-c0cf-4c3c-8120-7df4bb09288b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:08.506926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5c00c-c0cf-4c3c-8120-7df4bb09288b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:08.516678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:08.524288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 318/631] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:08.524288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5c00c-c0cf-4c3c-8120-7df4bb09288b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:08.553691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 314/632] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:08.554340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb384ee-4ff0-433a-b747-d4b64ef0be0a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:08.602968 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:08.656994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5c00c-c0cf-4c3c-8120-7df4bb09288b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:08.658023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb384ee-4ff0-433a-b747-d4b64ef0be0a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:08.691600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d945d7-7122-42fe-99f4-84ae420aaa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:08.692246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d945d7-7122-42fe-99f4-84ae420aaa51 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.692530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d945d7-7122-42fe-99f4-84ae420aaa51 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.692784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d945d7-7122-42fe-99f4-84ae420aaa51 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.695711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a9fcb-1f9d-4471-b4f8-de13bdabece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:08.696374 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a9fcb-1f9d-4471-b4f8-de13bdabece6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.696666 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a9fcb-1f9d-4471-b4f8-de13bdabece6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.696936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a9fcb-1f9d-4471-b4f8-de13bdabece6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.701538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d945d7-7122-42fe-99f4-84ae420aaa51 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:08.703223 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d945d7-7122-42fe-99f4-84ae420aaa51 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:08.707212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a9fcb-1f9d-4471-b4f8-de13bdabece6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:08.708676 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a9fcb-1f9d-4471-b4f8-de13bdabece6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-01d945d7-7122-42fe-99f4-84ae420aaa51 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:08.717379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:08.718329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:08.719249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:08.719249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:08.719249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:08.719249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:08.719249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:08.719249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:08.719249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:08.719249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:08.719249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:08.719249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:08.719808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 319/633] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:08.720464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d945d7-7122-42fe-99f4-84ae420aaa51 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-256a9fcb-1f9d-4471-b4f8-de13bdabece6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:08.729073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:08.729961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:08.735881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:08.735881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:08.735881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:08.735881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:08.735881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 315/634] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:08.735881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a9fcb-1f9d-4471-b4f8-de13bdabece6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:08.761710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:08.826541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d945d7-7122-42fe-99f4-84ae420aaa51 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:08.838062 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:08.839411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a9fcb-1f9d-4471-b4f8-de13bdabece6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:08.864429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f188696-31c3-4991-8969-b122f3e1ee9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:08.865040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f188696-31c3-4991-8969-b122f3e1ee9c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.865327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f188696-31c3-4991-8969-b122f3e1ee9c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.865604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f188696-31c3-4991-8969-b122f3e1ee9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.866389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f188696-31c3-4991-8969-b122f3e1ee9c None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=4eedd711c6114de1bfcdf692ce058794) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:08.866697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f188696-31c3-4991-8969-b122f3e1ee9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:08.868461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f188696-31c3-4991-8969-b122f3e1ee9c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:08.873424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa601d26-f098-4998-8b4f-110fb887f2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:08.874054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa601d26-f098-4998-8b4f-110fb887f2ab None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:08.874332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa601d26-f098-4998-8b4f-110fb887f2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:08.874609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa601d26-f098-4998-8b4f-110fb887f2ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:08.875010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 320/635] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:08 2022] GET /identity/v3/role_assignments?user.id=4eedd711c6114de1bfcdf692ce058794&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:08.875772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f188696-31c3-4991-8969-b122f3e1ee9c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:08.893236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa601d26-f098-4998-8b4f-110fb887f2ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-fa601d26-f098-4998-8b4f-110fb887f2ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:08.899133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:08.900205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 316/636] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:08.901659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa601d26-f098-4998-8b4f-110fb887f2ab None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:08.998915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f188696-31c3-4991-8969-b122f3e1ee9c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:09.009460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:09.027177 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa601d26-f098-4998-8b4f-110fb887f2ab None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:09.036569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f0f0f-f9fc-4828-96eb-e5491d99284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:09.037451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f0f0f-f9fc-4828-96eb-e5491d99284d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.037451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f0f0f-f9fc-4828-96eb-e5491d99284d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.037730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f0f0f-f9fc-4828-96eb-e5491d99284d None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.039040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f0f0f-f9fc-4828-96eb-e5491d99284d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:09.039040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f0f0f-f9fc-4828-96eb-e5491d99284d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:09.040768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f0f0f-f9fc-4828-96eb-e5491d99284d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:09.058465 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 321/637] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Aug 4 07:24:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:09.060029 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d142e81-1cca-424e-a0d2-b59255424208 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.060312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d142e81-1cca-424e-a0d2-b59255424208 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.060590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d142e81-1cca-424e-a0d2-b59255424208 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.061938 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 322/638] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:09.062553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d142e81-1cca-424e-a0d2-b59255424208 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:09.067673 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca324e-5444-4d99-a5e2-5fd02aecff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:09.068396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca324e-5444-4d99-a5e2-5fd02aecff1d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.068684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca324e-5444-4d99-a5e2-5fd02aecff1d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.068956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca324e-5444-4d99-a5e2-5fd02aecff1d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.069657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca324e-5444-4d99-a5e2-5fd02aecff1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:09.069962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca324e-5444-4d99-a5e2-5fd02aecff1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:09.071348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca324e-5444-4d99-a5e2-5fd02aecff1d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:09.084898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 317/639] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:09.085726 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca324e-5444-4d99-a5e2-5fd02aecff1d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:09.098551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:09.175949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d142e81-1cca-424e-a0d2-b59255424208 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:09.189911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca324e-5444-4d99-a5e2-5fd02aecff1d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:09.212429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbe096-9f28-4430-a9ab-2167bf9cfbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:09.212965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbe096-9f28-4430-a9ab-2167bf9cfbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.213249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbe096-9f28-4430-a9ab-2167bf9cfbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.213674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbe096-9f28-4430-a9ab-2167bf9cfbc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.214325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbe096-9f28-4430-a9ab-2167bf9cfbc1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:09.214542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbe096-9f28-4430-a9ab-2167bf9cfbc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:09.215895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbe096-9f28-4430-a9ab-2167bf9cfbc1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:09.228823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 323/640] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:09.228823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70ed9c-b4f9-4d77-9911-e1705483face None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.228823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70ed9c-b4f9-4d77-9911-e1705483face None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.228823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70ed9c-b4f9-4d77-9911-e1705483face None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.230371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f85b2-1358-4960-9aa8-715908516135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:09.231020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f85b2-1358-4960-9aa8-715908516135 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.231303 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f85b2-1358-4960-9aa8-715908516135 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.231575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f85b2-1358-4960-9aa8-715908516135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.243506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f85b2-1358-4960-9aa8-715908516135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:09.245000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f85b2-1358-4960-9aa8-715908516135 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:09.246073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ee70ed9c-b4f9-4d77-9911-e1705483face None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:09.247255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ee70ed9c-b4f9-4d77-9911-e1705483face None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:09.252531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8f8f85b2-1358-4960-9aa8-715908516135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:09.261124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:09.262043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:09.262926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:09.262926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:09.262926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:09.262926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:09.262926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 318/641] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:09.263216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f85b2-1358-4960-9aa8-715908516135 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:09.336049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 324/642] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:09.336049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70ed9c-b4f9-4d77-9911-e1705483face None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:09.353417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:09.381898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f85b2-1358-4960-9aa8-715908516135 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:09.417207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a98ef5-f1e3-4d39-a011-d748507c43ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:09.417943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a98ef5-f1e3-4d39-a011-d748507c43ed None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.418281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a98ef5-f1e3-4d39-a011-d748507c43ed None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.418621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a98ef5-f1e3-4d39-a011-d748507c43ed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.426904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a98ef5-f1e3-4d39-a011-d748507c43ed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:09.428443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a98ef5-f1e3-4d39-a011-d748507c43ed None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-88a98ef5-f1e3-4d39-a011-d748507c43ed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:09.439561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:09.441337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:09.442530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:09.442530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:09.442530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:09.442530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:09.442530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:09.442530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:09.442530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:09.442530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:09.442530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 07:24:09.442530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:09.443396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70ed9c-b4f9-4d77-9911-e1705483face None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:09.447575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 319/643] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:09.449136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a98ef5-f1e3-4d39-a011-d748507c43ed None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:09.483678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fdc04-9d9d-496f-8270-cd8129b68667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:09.484287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fdc04-9d9d-496f-8270-cd8129b68667 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.484545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fdc04-9d9d-496f-8270-cd8129b68667 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.484831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fdc04-9d9d-496f-8270-cd8129b68667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.495481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fdc04-9d9d-496f-8270-cd8129b68667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:09.496959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fdc04-9d9d-496f-8270-cd8129b68667 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:09.509236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d85fdc04-9d9d-496f-8270-cd8129b68667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:09.512672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:09.513551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:09.514593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:09.514593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:09.514593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:09.514593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:09.514593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 325/644] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:09.514593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fdc04-9d9d-496f-8270-cd8129b68667 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:09.562354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a98ef5-f1e3-4d39-a011-d748507c43ed None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:09.596732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e6169-db85-41e1-894a-323f01cc732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:09.597900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e6169-db85-41e1-894a-323f01cc732f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.598243 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e6169-db85-41e1-894a-323f01cc732f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.598527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e6169-db85-41e1-894a-323f01cc732f None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.599027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:09.599213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e6169-db85-41e1-894a-323f01cc732f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:09.599519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e6169-db85-41e1-894a-323f01cc732f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:09.600924 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e6169-db85-41e1-894a-323f01cc732f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:09.617182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 320/645] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:09.617182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e6169-db85-41e1-894a-323f01cc732f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:09.627443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fdc04-9d9d-496f-8270-cd8129b68667 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:09.665053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546ba4c-a7e0-4b69-a738-840ae24adf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:09.665760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546ba4c-a7e0-4b69-a738-840ae24adf30 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.666095 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546ba4c-a7e0-4b69-a738-840ae24adf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.666473 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546ba4c-a7e0-4b69-a738-840ae24adf30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.678095 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546ba4c-a7e0-4b69-a738-840ae24adf30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d546ba4c-a7e0-4b69-a738-840ae24adf30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:09.683936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:09.684879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:09.685705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 326/646] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:09.686428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546ba4c-a7e0-4b69-a738-840ae24adf30 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:09.722385 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e6169-db85-41e1-894a-323f01cc732f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:09.739354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:09.761049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0bcbe-9fc5-4fb3-b92f-5548261f09e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:09.761868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0bcbe-9fc5-4fb3-b92f-5548261f09e3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.762339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0bcbe-9fc5-4fb3-b92f-5548261f09e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.762618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0bcbe-9fc5-4fb3-b92f-5548261f09e3 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.763418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0bcbe-9fc5-4fb3-b92f-5548261f09e3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:09.763724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0bcbe-9fc5-4fb3-b92f-5548261f09e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:09.765181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0bcbe-9fc5-4fb3-b92f-5548261f09e3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:09.785749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 321/647] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Aug 4 07:24:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:09.786638 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0bcbe-9fc5-4fb3-b92f-5548261f09e3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:09.813753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546ba4c-a7e0-4b69-a738-840ae24adf30 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:09.849152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00f736-7982-413a-b78e-118ef08c4565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:09.850021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00f736-7982-413a-b78e-118ef08c4565 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.850510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00f736-7982-413a-b78e-118ef08c4565 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.850962 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00f736-7982-413a-b78e-118ef08c4565 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.851804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00f736-7982-413a-b78e-118ef08c4565 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:09.852294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00f736-7982-413a-b78e-118ef08c4565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:09.853793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00f736-7982-413a-b78e-118ef08c4565 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:09.856651 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:09.871660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 327/648] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:09.871660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00f736-7982-413a-b78e-118ef08c4565 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:09.895731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0bcbe-9fc5-4fb3-b92f-5548261f09e3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:09.931999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46225d7-9e7d-4929-8f1b-6f29e52fdfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:09.932637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46225d7-9e7d-4929-8f1b-6f29e52fdfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:09.932926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46225d7-9e7d-4929-8f1b-6f29e52fdfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:09.933212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46225d7-9e7d-4929-8f1b-6f29e52fdfb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:09.939866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46225d7-9e7d-4929-8f1b-6f29e52fdfb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:09.941500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46225d7-9e7d-4929-8f1b-6f29e52fdfb3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f46225d7-9e7d-4929-8f1b-6f29e52fdfb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:09.955268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:09.956488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:09.957639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:09.957639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:09.957639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:09.957639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:09.957639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:09.957639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:09.957639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:09.957639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:09.961443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 322/649] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:09.962624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46225d7-9e7d-4929-8f1b-6f29e52fdfb3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:09.980895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00f736-7982-413a-b78e-118ef08c4565 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:10.008712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:10.034446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0d5fc-16d6-42fd-aa3b-ae0a9ab24fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:10.035418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d5fc-16d6-42fd-aa3b-ae0a9ab24fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:10.035885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d5fc-16d6-42fd-aa3b-ae0a9ab24fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:10.036380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d5fc-16d6-42fd-aa3b-ae0a9ab24fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:10.037581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b0d5fc-16d6-42fd-aa3b-ae0a9ab24fb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:10.037995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b0d5fc-16d6-42fd-aa3b-ae0a9ab24fb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:10.040369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b0d5fc-16d6-42fd-aa3b-ae0a9ab24fb7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:10.058698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 328/650] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:10.058698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0d5fc-16d6-42fd-aa3b-ae0a9ab24fb7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:10.098427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46225d7-9e7d-4929-8f1b-6f29e52fdfb3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:10.122778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:10.134783 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022cd20-a110-43b8-bbc9-c907f0269859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:10.135421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022cd20-a110-43b8-bbc9-c907f0269859 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:10.135707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022cd20-a110-43b8-bbc9-c907f0269859 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:10.135993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022cd20-a110-43b8-bbc9-c907f0269859 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:10.169819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0d5fc-16d6-42fd-aa3b-ae0a9ab24fb7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:10.208628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022cd20-a110-43b8-bbc9-c907f0269859 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:10.208815 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91584de-e231-4f82-a0bd-ce3fbcbf1ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:10.209266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91584de-e231-4f82-a0bd-ce3fbcbf1ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:10.209547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91584de-e231-4f82-a0bd-ce3fbcbf1ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:10.209821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91584de-e231-4f82-a0bd-ce3fbcbf1ae0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:10.210083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022cd20-a110-43b8-bbc9-c907f0269859 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:10.218144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91584de-e231-4f82-a0bd-ce3fbcbf1ae0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:10.219660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91584de-e231-4f82-a0bd-ce3fbcbf1ae0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b91584de-e231-4f82-a0bd-ce3fbcbf1ae0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:10.231830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:10.233258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 329/651] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:10.239828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91584de-e231-4f82-a0bd-ce3fbcbf1ae0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:10.255039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:10.331078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 323/652] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:24:10.331078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022cd20-a110-43b8-bbc9-c907f0269859 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:10.341647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91584de-e231-4f82-a0bd-ce3fbcbf1ae0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:10.358348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:10.375086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6aca5-328a-46a7-986a-225eadc4b6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:10.375827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6aca5-328a-46a7-986a-225eadc4b6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:10.376197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6aca5-328a-46a7-986a-225eadc4b6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:10.376544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6aca5-328a-46a7-986a-225eadc4b6b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:10.377487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6aca5-328a-46a7-986a-225eadc4b6b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:10.377859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6aca5-328a-46a7-986a-225eadc4b6b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:10.379882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6aca5-328a-46a7-986a-225eadc4b6b0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:10.392718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 330/653] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:10.393570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6aca5-328a-46a7-986a-225eadc4b6b0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:10.442959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022cd20-a110-43b8-bbc9-c907f0269859 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:10.480115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc4622-71f7-42cd-9d17-0a0f646a0f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:10.480858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc4622-71f7-42cd-9d17-0a0f646a0f8a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:10.481200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc4622-71f7-42cd-9d17-0a0f646a0f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:10.481539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc4622-71f7-42cd-9d17-0a0f646a0f8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:10.483892 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:10.494230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc4622-71f7-42cd-9d17-0a0f646a0f8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:10.495786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc4622-71f7-42cd-9d17-0a0f646a0f8a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:10.497303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6aca5-328a-46a7-986a-225eadc4b6b0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-69fc4622-71f7-42cd-9d17-0a0f646a0f8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:10.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:10.512619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:10.513490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:10.513490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:10.513490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:10.513490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:10.513490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 324/654] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:10.514178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc4622-71f7-42cd-9d17-0a0f646a0f8a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:10.539208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1aed11-4fb5-4371-9846-f69778060cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:10.539826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1aed11-4fb5-4371-9846-f69778060cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:10.540143 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1aed11-4fb5-4371-9846-f69778060cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:10.540422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1aed11-4fb5-4371-9846-f69778060cd0 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:10.541215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1aed11-4fb5-4371-9846-f69778060cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:10.541523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1aed11-4fb5-4371-9846-f69778060cd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:10.542953 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1aed11-4fb5-4371-9846-f69778060cd0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:10.561004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 331/655] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Aug 4 07:24:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:10.561787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1aed11-4fb5-4371-9846-f69778060cd0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:10.598582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:10.624343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc4622-71f7-42cd-9d17-0a0f646a0f8a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:10.659832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6b67b-083d-423d-9bbd-b66bf0381b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:10.660530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6b67b-083d-423d-9bbd-b66bf0381b2a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:10.660813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6b67b-083d-423d-9bbd-b66bf0381b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:10.661090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6b67b-083d-423d-9bbd-b66bf0381b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:10.666858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1aed11-4fb5-4371-9846-f69778060cd0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:10.672027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6b67b-083d-423d-9bbd-b66bf0381b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:10.673475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6b67b-083d-423d-9bbd-b66bf0381b2a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-83f6b67b-083d-423d-9bbd-b66bf0381b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:10.688105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:10.689028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:10.689907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:10.689907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:10.689907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:10.689907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:10.689907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 325/656] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:10.690192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6b67b-083d-423d-9bbd-b66bf0381b2a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:10.706701 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bb34a-6190-4292-8c1d-a74f2eeec083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:10.707368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bb34a-6190-4292-8c1d-a74f2eeec083 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:10.707655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bb34a-6190-4292-8c1d-a74f2eeec083 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:10.707932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bb34a-6190-4292-8c1d-a74f2eeec083 None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/9503e2888d5e4831afc0d9d9522c3eea/roles/88637a5a7c9d47aaacf803a09abc1755` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:10.722919 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bb34a-6190-4292-8c1d-a74f2eeec083 None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=9503e2888d5e4831afc0d9d9522c3eea, role_id=88637a5a7c9d47aaacf803a09abc1755)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:10.724366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bb34a-6190-4292-8c1d-a74f2eeec083 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:10.731451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:10.791134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6b67b-083d-423d-9bbd-b66bf0381b2a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:10.800754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 332/657] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:24:10 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/9503e2888d5e4831afc0d9d9522c3eea/roles/88637a5a7c9d47aaacf803a09abc1755 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:24:10.801799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bb34a-6190-4292-8c1d-a74f2eeec083 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:10.842940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f376e01-a364-4746-9f57-c94bfcecf32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:10.843818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f376e01-a364-4746-9f57-c94bfcecf32b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:10.844258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f376e01-a364-4746-9f57-c94bfcecf32b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:10.844676 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f376e01-a364-4746-9f57-c94bfcecf32b None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:10.845652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f376e01-a364-4746-9f57-c94bfcecf32b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:10.846063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f376e01-a364-4746-9f57-c94bfcecf32b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:10.848829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f376e01-a364-4746-9f57-c94bfcecf32b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:10.864716 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 326/658] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:10.865857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f376e01-a364-4746-9f57-c94bfcecf32b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:10.924125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:10.945278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bb34a-6190-4292-8c1d-a74f2eeec083 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:10.993262 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede3777-fe51-4d3b-9086-d7c8d5bd3237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:10.994326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede3777-fe51-4d3b-9086-d7c8d5bd3237 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:10.994326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede3777-fe51-4d3b-9086-d7c8d5bd3237 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:10.994516 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede3777-fe51-4d3b-9086-d7c8d5bd3237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.006068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:11.010617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f376e01-a364-4746-9f57-c94bfcecf32b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:11.050097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06770a7e-9c54-44f3-9b1a-6a252205af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:11.050880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06770a7e-9c54-44f3-9b1a-6a252205af44 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:11.051166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06770a7e-9c54-44f3-9b1a-6a252205af44 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:11.051964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06770a7e-9c54-44f3-9b1a-6a252205af44 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.052758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06770a7e-9c54-44f3-9b1a-6a252205af44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:11.053225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06770a7e-9c54-44f3-9b1a-6a252205af44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:11.054619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06770a7e-9c54-44f3-9b1a-6a252205af44 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:11.070587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 327/659] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:11.070587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06770a7e-9c54-44f3-9b1a-6a252205af44 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:11.073581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede3777-fe51-4d3b-9086-d7c8d5bd3237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:11.075060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede3777-fe51-4d3b-9086-d7c8d5bd3237 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:11.163409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:11.172402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06770a7e-9c54-44f3-9b1a-6a252205af44 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:11.180090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 333/660] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:24:11.181347 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4555c-13bf-4189-aa58-655496f2c7bf None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:11.181637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4555c-13bf-4189-aa58-655496f2c7bf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:11.181918 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4555c-13bf-4189-aa58-655496f2c7bf None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.183376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 334/661] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:11.183939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4555c-13bf-4189-aa58-655496f2c7bf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:11.215707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ece274-90f1-4914-8315-4da232a62774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:11.216375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ece274-90f1-4914-8315-4da232a62774 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:11.216656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ece274-90f1-4914-8315-4da232a62774 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:11.216933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ece274-90f1-4914-8315-4da232a62774 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.225202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ece274-90f1-4914-8315-4da232a62774 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:11.226637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ece274-90f1-4914-8315-4da232a62774 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-73ece274-90f1-4914-8315-4da232a62774 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:11.238226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:11.239388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:11.240633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:11.240633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:11.240633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:11.240633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:11.240633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:11.240633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:11.240633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:11.240633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:11.240633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 07:24:11.240633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:11.243886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 328/662] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:11.245068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18469a9-6be2-44bc-b574-4eb26cfd7171 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:11.245362 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18469a9-6be2-44bc-b574-4eb26cfd7171 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:11.245639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18469a9-6be2-44bc-b574-4eb26cfd7171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.263252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e18469a9-6be2-44bc-b574-4eb26cfd7171 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:11.264474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e18469a9-6be2-44bc-b574-4eb26cfd7171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:11.279222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:11.301349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4555c-13bf-4189-aa58-655496f2c7bf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:11.338414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1d411-2a60-4ef5-aa3d-3f780d7795c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:11.339147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1d411-2a60-4ef5-aa3d-3f780d7795c5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:11.339499 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1d411-2a60-4ef5-aa3d-3f780d7795c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:11.339777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1d411-2a60-4ef5-aa3d-3f780d7795c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.340602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 329/663] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:11.341439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18469a9-6be2-44bc-b574-4eb26cfd7171 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:11.357201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1d411-2a60-4ef5-aa3d-3f780d7795c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:11.359639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1d411-2a60-4ef5-aa3d-3f780d7795c5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-84e1d411-2a60-4ef5-aa3d-3f780d7795c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:11.375866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:11.376967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:11.378601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:11.378601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:11.378601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:11.378601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:11.378601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:11.378601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:11.378601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:11.378601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:11.378601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 335/664] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:11.379578 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1d411-2a60-4ef5-aa3d-3f780d7795c5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:11.434580 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:11.447795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18469a9-6be2-44bc-b574-4eb26cfd7171 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:11.484845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb39b1-28d8-4ae8-88d0-e2434072e4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:11.485491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb39b1-28d8-4ae8-88d0-e2434072e4fe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:11.485775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb39b1-28d8-4ae8-88d0-e2434072e4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:11.486053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb39b1-28d8-4ae8-88d0-e2434072e4fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.496056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb39b1-28d8-4ae8-88d0-e2434072e4fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:11.497468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1d411-2a60-4ef5-aa3d-3f780d7795c5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:11.497901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb39b1-28d8-4ae8-88d0-e2434072e4fe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:11.506559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-bacb39b1-28d8-4ae8-88d0-e2434072e4fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:11.514046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:11.521138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:11.522059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:11.522059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:11.522059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:11.522059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:11.522059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 330/665] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:11.522059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb39b1-28d8-4ae8-88d0-e2434072e4fe None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:11.541388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07219c42-87d7-457c-ab7a-778031ae0484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:11.542007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07219c42-87d7-457c-ab7a-778031ae0484 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:11.542291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07219c42-87d7-457c-ab7a-778031ae0484 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:11.542994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07219c42-87d7-457c-ab7a-778031ae0484 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.543356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07219c42-87d7-457c-ab7a-778031ae0484 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:11.543665 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07219c42-87d7-457c-ab7a-778031ae0484 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:11.545139 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07219c42-87d7-457c-ab7a-778031ae0484 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:11.563055 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 336/666] 10.222.0.64 () {56 vars in 1139 bytes} [Thu Aug 4 07:24:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:11.568953 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07219c42-87d7-457c-ab7a-778031ae0484 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:11.625303 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb39b1-28d8-4ae8-88d0-e2434072e4fe None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:11.662892 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109da197-0d27-4c3b-b082-f384292040f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:11.663531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109da197-0d27-4c3b-b082-f384292040f6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:11.663815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109da197-0d27-4c3b-b082-f384292040f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:11.664124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109da197-0d27-4c3b-b082-f384292040f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.671573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07219c42-87d7-457c-ab7a-778031ae0484 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:11.674801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:11.674937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109da197-0d27-4c3b-b082-f384292040f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-109da197-0d27-4c3b-b082-f384292040f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:11.680564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:11.681504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:11.682364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:11.689441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 331/667] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:11.689441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109da197-0d27-4c3b-b082-f384292040f6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:11.711375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02f249-1126-420b-8215-2eaa7c4122f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:11.712009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02f249-1126-420b-8215-2eaa7c4122f0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:11.712298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02f249-1126-420b-8215-2eaa7c4122f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:11.712569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02f249-1126-420b-8215-2eaa7c4122f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.713372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02f249-1126-420b-8215-2eaa7c4122f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:11.713683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02f249-1126-420b-8215-2eaa7c4122f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:11.715101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02f249-1126-420b-8215-2eaa7c4122f0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:11.726812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 337/668] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:11.727594 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02f249-1126-420b-8215-2eaa7c4122f0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:11.752570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:11.796074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109da197-0d27-4c3b-b082-f384292040f6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:11.831132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20da20-a628-4ff3-b9b1-9b6e809b9e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:11.831801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20da20-a628-4ff3-b9b1-9b6e809b9e0d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:11.832132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20da20-a628-4ff3-b9b1-9b6e809b9e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:11.832412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20da20-a628-4ff3-b9b1-9b6e809b9e0d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.832797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02f249-1126-420b-8215-2eaa7c4122f0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:11.833104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20da20-a628-4ff3-b9b1-9b6e809b9e0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:11.833408 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20da20-a628-4ff3-b9b1-9b6e809b9e0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:11.835677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20da20-a628-4ff3-b9b1-9b6e809b9e0d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:11.853556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 332/669] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:11.854419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20da20-a628-4ff3-b9b1-9b6e809b9e0d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:11.867883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90d582-f73d-4c6e-9d65-31af95063869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:11.868534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90d582-f73d-4c6e-9d65-31af95063869 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:11.868821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90d582-f73d-4c6e-9d65-31af95063869 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:11.869102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90d582-f73d-4c6e-9d65-31af95063869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:11.902060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:11.942340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90d582-f73d-4c6e-9d65-31af95063869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:11.943859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90d582-f73d-4c6e-9d65-31af95063869 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:11.971593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20da20-a628-4ff3-b9b1-9b6e809b9e0d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:11.995104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:12.011905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c69a5a-3e98-4a4e-a9ca-c44de1422adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:12.012552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c69a5a-3e98-4a4e-a9ca-c44de1422adb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:12.012855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c69a5a-3e98-4a4e-a9ca-c44de1422adb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:12.013114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c69a5a-3e98-4a4e-a9ca-c44de1422adb None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:12.013828 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c69a5a-3e98-4a4e-a9ca-c44de1422adb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:12.014135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c69a5a-3e98-4a4e-a9ca-c44de1422adb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:12.015504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c69a5a-3e98-4a4e-a9ca-c44de1422adb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:12.024411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 333/670] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:12.025190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c69a5a-3e98-4a4e-a9ca-c44de1422adb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:12.061435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 338/671] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:24:12.062281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90d582-f73d-4c6e-9d65-31af95063869 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:12.126603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:12.137795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c69a5a-3e98-4a4e-a9ca-c44de1422adb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:12.165138 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90d582-f73d-4c6e-9d65-31af95063869 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:12.172405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96000868-28df-48c7-b586-d6df000f118e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:12.173485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000868-28df-48c7-b586-d6df000f118e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:12.173834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000868-28df-48c7-b586-d6df000f118e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:12.174175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000868-28df-48c7-b586-d6df000f118e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:12.175460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96000868-28df-48c7-b586-d6df000f118e None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=eab096d4ccae4bc09e6177afd85a55de) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:12.175827 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96000868-28df-48c7-b586-d6df000f118e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:12.177604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96000868-28df-48c7-b586-d6df000f118e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:12.192013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 334/672] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:12 2022] GET /identity/v3/role_assignments?user.id=eab096d4ccae4bc09e6177afd85a55de&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:12.192013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96000868-28df-48c7-b586-d6df000f118e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:12.214932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4031a-e66d-49e7-9a40-0fc43720c80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:12.215650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4031a-e66d-49e7-9a40-0fc43720c80c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:12.216029 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4031a-e66d-49e7-9a40-0fc43720c80c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:12.216432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4031a-e66d-49e7-9a40-0fc43720c80c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:12.224785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4031a-e66d-49e7-9a40-0fc43720c80c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:12.226377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4031a-e66d-49e7-9a40-0fc43720c80c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d5f4031a-e66d-49e7-9a40-0fc43720c80c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:12.240177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:12.246418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:12.247521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:12.247521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:12.247521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:12.247521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:12.247521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:12.247521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:12.247521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:12.247521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:12.247521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 339/673] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:12.249384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4031a-e66d-49e7-9a40-0fc43720c80c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:12.251505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:12.326521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96000868-28df-48c7-b586-d6df000f118e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:12.374387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329253f-8b33-470a-aa14-1c3dd95a7665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:12.375124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329253f-8b33-470a-aa14-1c3dd95a7665 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:12.375414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329253f-8b33-470a-aa14-1c3dd95a7665 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:12.375691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329253f-8b33-470a-aa14-1c3dd95a7665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:12.382071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4031a-e66d-49e7-9a40-0fc43720c80c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:12.387471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329253f-8b33-470a-aa14-1c3dd95a7665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:12.388966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329253f-8b33-470a-aa14-1c3dd95a7665 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f329253f-8b33-470a-aa14-1c3dd95a7665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:12.408439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:12.409489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:12.410508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:12.410508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:12.410508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:12.410508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:12.410508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 335/674] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:12.410508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329253f-8b33-470a-aa14-1c3dd95a7665 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:12.424399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:12.439541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2316f-a4e0-46ba-95a2-8386259a5536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:12.443255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2316f-a4e0-46ba-95a2-8386259a5536 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:12.443255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2316f-a4e0-46ba-95a2-8386259a5536 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:12.443255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2316f-a4e0-46ba-95a2-8386259a5536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:12.454066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2316f-a4e0-46ba-95a2-8386259a5536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:12.457007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2316f-a4e0-46ba-95a2-8386259a5536 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-dce2316f-a4e0-46ba-95a2-8386259a5536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:12.476823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:12.477928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:12.479032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:12.479032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:12.479032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:12.479032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:12.479032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 340/675] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:12.484009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2316f-a4e0-46ba-95a2-8386259a5536 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:12.530897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:12.535587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329253f-8b33-470a-aa14-1c3dd95a7665 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:12.570867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecb262-b613-4b73-84ae-d0e22efd9d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:12.571487 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb262-b613-4b73-84ae-d0e22efd9d55 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:12.571765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb262-b613-4b73-84ae-d0e22efd9d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:12.572077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb262-b613-4b73-84ae-d0e22efd9d55 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:12.572894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecb262-b613-4b73-84ae-d0e22efd9d55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:12.573197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecb262-b613-4b73-84ae-d0e22efd9d55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:12.574731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecb262-b613-4b73-84ae-d0e22efd9d55 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:12.589446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 336/676] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:12.590282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecb262-b613-4b73-84ae-d0e22efd9d55 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:12.604978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2316f-a4e0-46ba-95a2-8386259a5536 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:12.643065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59240ae1-943a-4190-bc60-a7df6d164510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:12.643898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59240ae1-943a-4190-bc60-a7df6d164510 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:12.644302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59240ae1-943a-4190-bc60-a7df6d164510 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:12.644678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59240ae1-943a-4190-bc60-a7df6d164510 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:12.645495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59240ae1-943a-4190-bc60-a7df6d164510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:12.645969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59240ae1-943a-4190-bc60-a7df6d164510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:12.647484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59240ae1-943a-4190-bc60-a7df6d164510 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:12.661921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 341/677] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:12.662977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59240ae1-943a-4190-bc60-a7df6d164510 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:12.703368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:12.777255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59240ae1-943a-4190-bc60-a7df6d164510 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:12.814980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ac13d-7aad-4524-a2e1-2575674b9e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:12.816623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ac13d-7aad-4524-a2e1-2575674b9e38 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:12.817139 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ac13d-7aad-4524-a2e1-2575674b9e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:12.817668 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ac13d-7aad-4524-a2e1-2575674b9e38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:12.818821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ac13d-7aad-4524-a2e1-2575674b9e38 None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=56dfc4dfda424d40b7c9f17345b8ad45) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:12.819354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ac13d-7aad-4524-a2e1-2575674b9e38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:12.821124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ac13d-7aad-4524-a2e1-2575674b9e38 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:12.824834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecb262-b613-4b73-84ae-d0e22efd9d55 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:12.836528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 342/678] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:12 2022] GET /identity/v3/role_assignments?user.id=56dfc4dfda424d40b7c9f17345b8ad45&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:12.837877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ac13d-7aad-4524-a2e1-2575674b9e38 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:12.862815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef6496-1944-46e0-a0d7-1ebf4edc1649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:12.863484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef6496-1944-46e0-a0d7-1ebf4edc1649 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:12.868016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef6496-1944-46e0-a0d7-1ebf4edc1649 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:12.868016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef6496-1944-46e0-a0d7-1ebf4edc1649 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:12.868016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef6496-1944-46e0-a0d7-1ebf4edc1649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:12.868016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef6496-1944-46e0-a0d7-1ebf4edc1649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:12.868016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef6496-1944-46e0-a0d7-1ebf4edc1649 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:12.881604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 337/679] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:12.883331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc12634-b625-4d7a-8793-77db0d6c2276 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:12.883620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc12634-b625-4d7a-8793-77db0d6c2276 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:12.883893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc12634-b625-4d7a-8793-77db0d6c2276 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:12.885568 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 338/680] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:12.887735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc12634-b625-4d7a-8793-77db0d6c2276 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:12.916386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:12.975942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:12.979218 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ac13d-7aad-4524-a2e1-2575674b9e38 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:13.026884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30991394-84d9-4850-9428-a0ab0bee5051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:13.027682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991394-84d9-4850-9428-a0ab0bee5051 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:13.027909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991394-84d9-4850-9428-a0ab0bee5051 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:13.029358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991394-84d9-4850-9428-a0ab0bee5051 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:13.036394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc12634-b625-4d7a-8793-77db0d6c2276 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:13.038276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30991394-84d9-4850-9428-a0ab0bee5051 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:13.040702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30991394-84d9-4850-9428-a0ab0bee5051 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-30991394-84d9-4850-9428-a0ab0bee5051 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:13.055385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:13.056437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:13.057922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:13.057922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:13.057922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:13.057922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:13.057922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:13.057922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:13.057922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:13.057922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:13.057922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:13.057922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:13.057922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 343/681] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Aug 4 07:24:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:13.058531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9721d0b-6f75-4225-8afe-028cdb49e981 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:13.058797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9721d0b-6f75-4225-8afe-028cdb49e981 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:13.059073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9721d0b-6f75-4225-8afe-028cdb49e981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:13.076839 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49948caa-b98d-4cb4-8a13-71db1b076a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:13.077576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49948caa-b98d-4cb4-8a13-71db1b076a97 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:13.078279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49948caa-b98d-4cb4-8a13-71db1b076a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:13.078279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49948caa-b98d-4cb4-8a13-71db1b076a97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:13.082121 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e9721d0b-6f75-4225-8afe-028cdb49e981 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:13.083322 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e9721d0b-6f75-4225-8afe-028cdb49e981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:13.088820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49948caa-b98d-4cb4-8a13-71db1b076a97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:13.089779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49948caa-b98d-4cb4-8a13-71db1b076a97 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-49948caa-b98d-4cb4-8a13-71db1b076a97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:13.105706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:13.106629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:13.107798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:13.107798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:13.107798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:13.107798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:13.107798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:13.107798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:13.107798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:13.107798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:13.107798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 339/682] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:13.107798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49948caa-b98d-4cb4-8a13-71db1b076a97 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:13.168786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 344/683] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:13.170181 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9721d0b-6f75-4225-8afe-028cdb49e981 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:13.210246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:13.231462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49948caa-b98d-4cb4-8a13-71db1b076a97 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:13.267308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0018a2-4300-4618-861a-f6cd4eaaacdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:13.268063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0018a2-4300-4618-861a-f6cd4eaaacdd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:13.268454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0018a2-4300-4618-861a-f6cd4eaaacdd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:13.268901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0018a2-4300-4618-861a-f6cd4eaaacdd None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:13.269784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0018a2-4300-4618-861a-f6cd4eaaacdd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:13.270173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0018a2-4300-4618-861a-f6cd4eaaacdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:13.271708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0018a2-4300-4618-861a-f6cd4eaaacdd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:13.275441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:13.288487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9721d0b-6f75-4225-8afe-028cdb49e981 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:13.291166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 340/684] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Aug 4 07:24:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:13.291649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0018a2-4300-4618-861a-f6cd4eaaacdd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:13.326335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb8134-5acc-4654-a033-ff5124b205df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:13.328029 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb8134-5acc-4654-a033-ff5124b205df None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:13.328458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb8134-5acc-4654-a033-ff5124b205df None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:13.328876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb8134-5acc-4654-a033-ff5124b205df None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:13.329823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb8134-5acc-4654-a033-ff5124b205df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:13.330273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb8134-5acc-4654-a033-ff5124b205df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:13.333464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb8134-5acc-4654-a033-ff5124b205df None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:13.351757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 345/685] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:13.353195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb8134-5acc-4654-a033-ff5124b205df None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:13.402260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0018a2-4300-4618-861a-f6cd4eaaacdd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:13.449608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353d133-c6f3-4b52-a923-9e3db7836a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:13.450256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353d133-c6f3-4b52-a923-9e3db7836a0d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:13.450540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353d133-c6f3-4b52-a923-9e3db7836a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:13.450819 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353d133-c6f3-4b52-a923-9e3db7836a0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:13.454218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:13.466173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353d133-c6f3-4b52-a923-9e3db7836a0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e353d133-c6f3-4b52-a923-9e3db7836a0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:13.475368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:13.476596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:13.477596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 341/686] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:13.478322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353d133-c6f3-4b52-a923-9e3db7836a0d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:13.489955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb8134-5acc-4654-a033-ff5124b205df None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:13.535019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c18ef-b606-4f43-b86e-509e1a28860c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:13.535780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c18ef-b606-4f43-b86e-509e1a28860c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:13.535974 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c18ef-b606-4f43-b86e-509e1a28860c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:13.536338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c18ef-b606-4f43-b86e-509e1a28860c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:13.557447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:13.592557 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353d133-c6f3-4b52-a923-9e3db7836a0d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:13.612741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c18ef-b606-4f43-b86e-509e1a28860c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:13.614266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c18ef-b606-4f43-b86e-509e1a28860c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:13.630090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e440856-e074-440a-99c2-a327d53d960e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:13.630889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e440856-e074-440a-99c2-a327d53d960e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:13.631315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e440856-e074-440a-99c2-a327d53d960e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:13.631757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e440856-e074-440a-99c2-a327d53d960e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:13.632713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e440856-e074-440a-99c2-a327d53d960e None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=b80e701a81cb4fa893e55288559010f5) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:13.633164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e440856-e074-440a-99c2-a327d53d960e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:13.634723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e440856-e074-440a-99c2-a327d53d960e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:13.640978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 342/687] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:13 2022] GET /identity/v3/role_assignments?user.id=b80e701a81cb4fa893e55288559010f5&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:13.641914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e440856-e074-440a-99c2-a327d53d960e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:13.706169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:13.729067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 346/688] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:24:13.739072 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c18ef-b606-4f43-b86e-509e1a28860c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:13.756817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e440856-e074-440a-99c2-a327d53d960e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:13.795128 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157cc09-c133-430b-9b4b-0227f4b11944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:13.797014 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157cc09-c133-430b-9b4b-0227f4b11944 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:13.797479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157cc09-c133-430b-9b4b-0227f4b11944 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:13.798520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157cc09-c133-430b-9b4b-0227f4b11944 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:13.799729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157cc09-c133-430b-9b4b-0227f4b11944 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:13.802468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157cc09-c133-430b-9b4b-0227f4b11944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:13.804821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157cc09-c133-430b-9b4b-0227f4b11944 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:13.824193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 343/689] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:13.828964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157cc09-c133-430b-9b4b-0227f4b11944 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:13.835934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:13.918391 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c18ef-b606-4f43-b86e-509e1a28860c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:13.969625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1450c6b-a3e4-4a69-ae67-31400d19f04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:13.970530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1450c6b-a3e4-4a69-ae67-31400d19f04e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:13.970830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1450c6b-a3e4-4a69-ae67-31400d19f04e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:13.971530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1450c6b-a3e4-4a69-ae67-31400d19f04e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:13.979842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157cc09-c133-430b-9b4b-0227f4b11944 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:13.986634 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1450c6b-a3e4-4a69-ae67-31400d19f04e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:13.989265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1450c6b-a3e4-4a69-ae67-31400d19f04e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:14.007345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a1450c6b-a3e4-4a69-ae67-31400d19f04e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:14.011577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:14.012704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:14.013827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:14.013827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:14.013827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:14.013827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:14.015677 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 347/690] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:14.021327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1450c6b-a3e4-4a69-ae67-31400d19f04e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:14.023618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a083c2c-ff61-48c6-949b-8beb7f53b088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:14.024311 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a083c2c-ff61-48c6-949b-8beb7f53b088 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:14.024602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a083c2c-ff61-48c6-949b-8beb7f53b088 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:14.024867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a083c2c-ff61-48c6-949b-8beb7f53b088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:14.037596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a083c2c-ff61-48c6-949b-8beb7f53b088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:14.039044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a083c2c-ff61-48c6-949b-8beb7f53b088 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3a083c2c-ff61-48c6-949b-8beb7f53b088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:14.062474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:14.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:14.064868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:14.064868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:14.064868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:14.064868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:14.067241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 344/691] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:14.074324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a083c2c-ff61-48c6-949b-8beb7f53b088 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:14.154177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1450c6b-a3e4-4a69-ae67-31400d19f04e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:14.156006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:14.222618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b6dfd-24dc-4351-a5b9-6ae3f4cb5081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:14.223246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b6dfd-24dc-4351-a5b9-6ae3f4cb5081 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:14.223527 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b6dfd-24dc-4351-a5b9-6ae3f4cb5081 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:14.223803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b6dfd-24dc-4351-a5b9-6ae3f4cb5081 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:14.228774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b6dfd-24dc-4351-a5b9-6ae3f4cb5081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:14.229099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b6dfd-24dc-4351-a5b9-6ae3f4cb5081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:14.231621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b6dfd-24dc-4351-a5b9-6ae3f4cb5081 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:14.235762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a083c2c-ff61-48c6-949b-8beb7f53b088 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:14.250822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 348/692] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:14.261024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b6dfd-24dc-4351-a5b9-6ae3f4cb5081 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:14.275904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e233fe-25ac-4b50-be92-0147b30b3377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:14.276570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e233fe-25ac-4b50-be92-0147b30b3377 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:14.276854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e233fe-25ac-4b50-be92-0147b30b3377 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:14.277135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e233fe-25ac-4b50-be92-0147b30b3377 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:14.277909 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e233fe-25ac-4b50-be92-0147b30b3377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:14.278221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e233fe-25ac-4b50-be92-0147b30b3377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:14.279688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e233fe-25ac-4b50-be92-0147b30b3377 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:14.295791 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 345/693] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:14.302033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e233fe-25ac-4b50-be92-0147b30b3377 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:14.378763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:14.454957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b6dfd-24dc-4351-a5b9-6ae3f4cb5081 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:14.470849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e233fe-25ac-4b50-be92-0147b30b3377 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:14.509859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28e76d-ab0d-480d-9ba2-f5559523438d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:14.510496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28e76d-ab0d-480d-9ba2-f5559523438d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:14.510777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28e76d-ab0d-480d-9ba2-f5559523438d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:14.511046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28e76d-ab0d-480d-9ba2-f5559523438d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:14.513051 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558760a2-95a1-49b7-9159-14644b191503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:14.514532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558760a2-95a1-49b7-9159-14644b191503 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:14.515042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558760a2-95a1-49b7-9159-14644b191503 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:14.515532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558760a2-95a1-49b7-9159-14644b191503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:14.519012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28e76d-ab0d-480d-9ba2-f5559523438d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:14.520928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28e76d-ab0d-480d-9ba2-f5559523438d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:14.525551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558760a2-95a1-49b7-9159-14644b191503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:14.527220 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558760a2-95a1-49b7-9159-14644b191503 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2e28e76d-ab0d-480d-9ba2-f5559523438d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:14.532774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:14.533808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:14.534832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:14.534832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:14.534832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:14.534832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:14.534832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:14.534832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:14.534832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:14.534832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:14.534832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:14.534832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:14.538887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 346/694] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:14.540569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e49474-107c-4640-bf9c-2d04f875f195 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:14.540895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e49474-107c-4640-bf9c-2d04f875f195 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:14.541139 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e49474-107c-4640-bf9c-2d04f875f195 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-558760a2-95a1-49b7-9159-14644b191503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:14.543831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:14.544734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:14.545614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:14.545614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:14.545614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:14.545614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:14.545614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:14.545614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:14.545614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:14.545614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:14.546457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 347/695] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:14.547080 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e49474-107c-4640-bf9c-2d04f875f195 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:14.548639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 349/696] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:14.553791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08235a-83d2-4ed4-a09d-8211a2c9cb19 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:14.553791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08235a-83d2-4ed4-a09d-8211a2c9cb19 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:14.554018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08235a-83d2-4ed4-a09d-8211a2c9cb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:14.573345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ce08235a-83d2-4ed4-a09d-8211a2c9cb19 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:14.574517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ce08235a-83d2-4ed4-a09d-8211a2c9cb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:14.593215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:14.660241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 350/697] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:14.661117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08235a-83d2-4ed4-a09d-8211a2c9cb19 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:14.677947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:14.687708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e49474-107c-4640-bf9c-2d04f875f195 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:14.734802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7f673-e06f-4e85-b3cd-5fdfd817bd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:14.736096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7f673-e06f-4e85-b3cd-5fdfd817bd15 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:14.736620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7f673-e06f-4e85-b3cd-5fdfd817bd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:14.737762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7f673-e06f-4e85-b3cd-5fdfd817bd15 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:14.739400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7f673-e06f-4e85-b3cd-5fdfd817bd15 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:14.739874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7f673-e06f-4e85-b3cd-5fdfd817bd15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:14.741988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7f673-e06f-4e85-b3cd-5fdfd817bd15 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:14.766355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 348/698] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Aug 4 07:24:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:14.768717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7f673-e06f-4e85-b3cd-5fdfd817bd15 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:14.790945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08235a-83d2-4ed4-a09d-8211a2c9cb19 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:14.827450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c08bb8-e497-4d76-b552-78a0c24796c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:14.828065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c08bb8-e497-4d76-b552-78a0c24796c2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:14.828443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c08bb8-e497-4d76-b552-78a0c24796c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:14.828813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c08bb8-e497-4d76-b552-78a0c24796c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:14.829645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c08bb8-e497-4d76-b552-78a0c24796c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:14.830047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c08bb8-e497-4d76-b552-78a0c24796c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:14.832666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c08bb8-e497-4d76-b552-78a0c24796c2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:14.850090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 351/699] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:14.850090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c08bb8-e497-4d76-b552-78a0c24796c2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:14.869776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:14.900917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7f673-e06f-4e85-b3cd-5fdfd817bd15 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:14.950549 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f9c06-4908-488b-b36e-c59f5c0edb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:14.951586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f9c06-4908-488b-b36e-c59f5c0edb93 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:14.952101 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f9c06-4908-488b-b36e-c59f5c0edb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:14.953150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f9c06-4908-488b-b36e-c59f5c0edb93 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:14.954286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f9c06-4908-488b-b36e-c59f5c0edb93 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:14.956827 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f9c06-4908-488b-b36e-c59f5c0edb93 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-055f9c06-4908-488b-b36e-c59f5c0edb93 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:14.974500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:14.975553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 07:24:14.976452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:14.979295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c08bb8-e497-4d76-b552-78a0c24796c2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:14.980702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 349/700] 10.222.0.64 () {56 vars in 1129 bytes} [Thu Aug 4 07:24:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:14.981558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f9c06-4908-488b-b36e-c59f5c0edb93 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:15.000912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:15.027164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62b988-7d2c-43c7-b304-33e6e679d308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:15.028421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62b988-7d2c-43c7-b304-33e6e679d308 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:15.028934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62b988-7d2c-43c7-b304-33e6e679d308 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:15.029558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62b988-7d2c-43c7-b304-33e6e679d308 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:15.086567 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f9c06-4908-488b-b36e-c59f5c0edb93 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:15.109512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62b988-7d2c-43c7-b304-33e6e679d308 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:15.111300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62b988-7d2c-43c7-b304-33e6e679d308 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:15.124204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dd9cd-f6d1-4e1d-a39e-d0b8517375fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:15.124877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd9cd-f6d1-4e1d-a39e-d0b8517375fc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:15.125160 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd9cd-f6d1-4e1d-a39e-d0b8517375fc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:15.125433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd9cd-f6d1-4e1d-a39e-d0b8517375fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:15.126275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dd9cd-f6d1-4e1d-a39e-d0b8517375fc None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=4eedd711c6114de1bfcdf692ce058794) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:15.126636 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dd9cd-f6d1-4e1d-a39e-d0b8517375fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:15.128165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dd9cd-f6d1-4e1d-a39e-d0b8517375fc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:15.128680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:15.140015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 350/701] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:14 2022] GET /identity/v3/role_assignments?user.id=4eedd711c6114de1bfcdf692ce058794&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:15.140015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dd9cd-f6d1-4e1d-a39e-d0b8517375fc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:15.257643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 352/702] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 07:24:15.259433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c290f5d-f296-427f-b46d-c2649e77aa00 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:15.259959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c290f5d-f296-427f-b46d-c2649e77aa00 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:15.260520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c290f5d-f296-427f-b46d-c2649e77aa00 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:15.262397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 353/703] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:15.264390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c290f5d-f296-427f-b46d-c2649e77aa00 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:15.280231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:15.304623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dd9cd-f6d1-4e1d-a39e-d0b8517375fc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:15.351357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533f0e6-c83e-4a86-b293-8313a4f10f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:15.352064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533f0e6-c83e-4a86-b293-8313a4f10f75 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:15.352415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533f0e6-c83e-4a86-b293-8313a4f10f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:15.352693 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533f0e6-c83e-4a86-b293-8313a4f10f75 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:15.353377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533f0e6-c83e-4a86-b293-8313a4f10f75 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:15.353683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533f0e6-c83e-4a86-b293-8313a4f10f75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:15.355046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533f0e6-c83e-4a86-b293-8313a4f10f75 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:15.359969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 351/704] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Aug 4 07:24:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:15.362852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2dc2e-7440-4f7c-b7c9-8c85a7dbef67 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:15.363370 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2dc2e-7440-4f7c-b7c9-8c85a7dbef67 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:15.363854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2dc2e-7440-4f7c-b7c9-8c85a7dbef67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:15.389618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5fb2dc2e-7440-4f7c-b7c9-8c85a7dbef67 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:15.391160 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5fb2dc2e-7440-4f7c-b7c9-8c85a7dbef67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:15.420595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c290f5d-f296-427f-b46d-c2649e77aa00 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:15.435776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:15.458570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cdb50-ad08-43d1-93f4-57c3f6ea3e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:15.459479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cdb50-ad08-43d1-93f4-57c3f6ea3e9d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:15.459961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cdb50-ad08-43d1-93f4-57c3f6ea3e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:15.460444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cdb50-ad08-43d1-93f4-57c3f6ea3e9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:15.468048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 352/705] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:15.468048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2dc2e-7440-4f7c-b7c9-8c85a7dbef67 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:15.486027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cdb50-ad08-43d1-93f4-57c3f6ea3e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:15.488765 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cdb50-ad08-43d1-93f4-57c3f6ea3e9d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ae9cdb50-ad08-43d1-93f4-57c3f6ea3e9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:15.514875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:15.515845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:15.516809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:15.516809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:15.516809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:15.516809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:15.524289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 354/706] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:15.525547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cdb50-ad08-43d1-93f4-57c3f6ea3e9d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:15.547898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:15.574189 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2dc2e-7440-4f7c-b7c9-8c85a7dbef67 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:15.612750 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771dcf0-80d7-4b4a-85ae-88afbb72abf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:15.613303 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771dcf0-80d7-4b4a-85ae-88afbb72abf5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:15.613592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771dcf0-80d7-4b4a-85ae-88afbb72abf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:15.613875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771dcf0-80d7-4b4a-85ae-88afbb72abf5 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:15.614567 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771dcf0-80d7-4b4a-85ae-88afbb72abf5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:15.614878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771dcf0-80d7-4b4a-85ae-88afbb72abf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:15.616330 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771dcf0-80d7-4b4a-85ae-88afbb72abf5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:15.628012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 353/707] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Aug 4 07:24:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:15.628012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771dcf0-80d7-4b4a-85ae-88afbb72abf5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:15.661807 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cdb50-ad08-43d1-93f4-57c3f6ea3e9d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:15.726999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cec12-f0bc-47bd-afe0-b2299f322a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:15.728003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cec12-f0bc-47bd-afe0-b2299f322a6f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:15.728502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cec12-f0bc-47bd-afe0-b2299f322a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:15.728928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cec12-f0bc-47bd-afe0-b2299f322a6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:15.741359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560cec12-f0bc-47bd-afe0-b2299f322a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:15.743171 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560cec12-f0bc-47bd-afe0-b2299f322a6f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-560cec12-f0bc-47bd-afe0-b2299f322a6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:15.759462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:15.760618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:15.761737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:15.761737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:15.761737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:15.761737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:15.763318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 355/708] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:15.770164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cec12-f0bc-47bd-afe0-b2299f322a6f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:15.797166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:15.844038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771dcf0-80d7-4b4a-85ae-88afbb72abf5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:15.882840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cec12-f0bc-47bd-afe0-b2299f322a6f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:15.890470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd43cc5-1f2f-4f35-acef-cdfe8337c637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:15.891910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd43cc5-1f2f-4f35-acef-cdfe8337c637 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:15.892454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd43cc5-1f2f-4f35-acef-cdfe8337c637 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:15.892947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd43cc5-1f2f-4f35-acef-cdfe8337c637 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:15.894910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd43cc5-1f2f-4f35-acef-cdfe8337c637 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:15.895225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd43cc5-1f2f-4f35-acef-cdfe8337c637 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:15.896641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd43cc5-1f2f-4f35-acef-cdfe8337c637 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:15.904420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:15.914506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 354/709] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:15.920022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd43cc5-1f2f-4f35-acef-cdfe8337c637 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:15.922944 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa7074-7e96-4863-83d6-27ad1c2ed239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:15.923913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa7074-7e96-4863-83d6-27ad1c2ed239 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:15.924533 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa7074-7e96-4863-83d6-27ad1c2ed239 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:15.925017 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa7074-7e96-4863-83d6-27ad1c2ed239 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:15.926173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa7074-7e96-4863-83d6-27ad1c2ed239 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:15.926785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa7074-7e96-4863-83d6-27ad1c2ed239 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:15.928423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa7074-7e96-4863-83d6-27ad1c2ed239 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:15.941897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 356/710] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:15.947723 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa7074-7e96-4863-83d6-27ad1c2ed239 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:16.069752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:16.083534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd43cc5-1f2f-4f35-acef-cdfe8337c637 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:16.109232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa7074-7e96-4863-83d6-27ad1c2ed239 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:16.149945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d3061-92a4-4fc5-af47-8fd67a392902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:16.150594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d3061-92a4-4fc5-af47-8fd67a392902 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:16.150900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d3061-92a4-4fc5-af47-8fd67a392902 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:16.151189 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d3061-92a4-4fc5-af47-8fd67a392902 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:16.159898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d3061-92a4-4fc5-af47-8fd67a392902 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:16.160664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625120a-8ece-4a61-ab9b-9b244c12fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:16.161433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d3061-92a4-4fc5-af47-8fd67a392902 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:16.161820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625120a-8ece-4a61-ab9b-9b244c12fd5a None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:16.162353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625120a-8ece-4a61-ab9b-9b244c12fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:16.162805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625120a-8ece-4a61-ab9b-9b244c12fd5a None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:16.164238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625120a-8ece-4a61-ab9b-9b244c12fd5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:16.165561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625120a-8ece-4a61-ab9b-9b244c12fd5a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9e4d3061-92a4-4fc5-af47-8fd67a392902 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:16.177021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:16.177977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:16.178946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:16.178946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:16.178946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:16.178946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:16.178946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:16.178946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:16.178946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:16.178946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:16.180186 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 355/711] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:16.181669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6934ef-2d86-41fb-88cd-07fc616ed329 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:16.182169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6934ef-2d86-41fb-88cd-07fc616ed329 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:16.182659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6934ef-2d86-41fb-88cd-07fc616ed329 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:16.184417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 356/712] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:16.189392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6934ef-2d86-41fb-88cd-07fc616ed329 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:16.253484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:16.264310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 357/713] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:24:15 2022] POST /identity/v3/users => generated 254 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:24:16.267154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b395-8670-4d9a-8a76-369cb8c3d73a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:16.267726 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b395-8670-4d9a-8a76-369cb8c3d73a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:16.269042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b395-8670-4d9a-8a76-369cb8c3d73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:16.294123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b053b395-8670-4d9a-8a76-369cb8c3d73a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:16.296637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b053b395-8670-4d9a-8a76-369cb8c3d73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:16.302788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6934ef-2d86-41fb-88cd-07fc616ed329 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:16.378646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98cc9c9-3a7d-4e2c-ba38-c9bf75909020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:16.379303 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cc9c9-3a7d-4e2c-ba38-c9bf75909020 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:16.379656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cc9c9-3a7d-4e2c-ba38-c9bf75909020 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:16.379948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cc9c9-3a7d-4e2c-ba38-c9bf75909020 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:16.380793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98cc9c9-3a7d-4e2c-ba38-c9bf75909020 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:16.381107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98cc9c9-3a7d-4e2c-ba38-c9bf75909020 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:16.382648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98cc9c9-3a7d-4e2c-ba38-c9bf75909020 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:16.382958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 358/714] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:16.410811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 357/715] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:16.417078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b395-8670-4d9a-8a76-369cb8c3d73a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:16.421810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98cc9c9-3a7d-4e2c-ba38-c9bf75909020 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:16.492334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:16.534610 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b395-8670-4d9a-8a76-369cb8c3d73a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:16.540241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98cc9c9-3a7d-4e2c-ba38-c9bf75909020 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:16.557754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:16.572142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c9d28-ef16-4b50-9f31-31ba569e25b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:16.573606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c9d28-ef16-4b50-9f31-31ba569e25b6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:16.574150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c9d28-ef16-4b50-9f31-31ba569e25b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:16.574714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c9d28-ef16-4b50-9f31-31ba569e25b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:16.575928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c9d28-ef16-4b50-9f31-31ba569e25b6 None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=9503e2888d5e4831afc0d9d9522c3eea) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:16.580220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908297c7-d876-4939-90be-9e837042c26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:16.580389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c9d28-ef16-4b50-9f31-31ba569e25b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:16.580501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908297c7-d876-4939-90be-9e837042c26e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:16.580501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908297c7-d876-4939-90be-9e837042c26e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:16.580501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908297c7-d876-4939-90be-9e837042c26e None admin] PATH_INFO: `/v3/services/network` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:16.581302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c9d28-ef16-4b50-9f31-31ba569e25b6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:16.581399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908297c7-d876-4939-90be-9e837042c26e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:16.581399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908297c7-d876-4939-90be-9e837042c26e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-908297c7-d876-4939-90be-9e837042c26e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:16.592086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:16.593144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 07:24:16.593985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:16.594716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 359/716] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:16 2022] GET /identity/v3/role_assignments?user.id=9503e2888d5e4831afc0d9d9522c3eea&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:16.597417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 358/717] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:16 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:16.603781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c9d28-ef16-4b50-9f31-31ba569e25b6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:16.719499 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c9d28-ef16-4b50-9f31-31ba569e25b6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:16.750205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:16.760021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47cf75-c882-420e-aae2-5ba74e909744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:16.760759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47cf75-c882-420e-aae2-5ba74e909744 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:16.761046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47cf75-c882-420e-aae2-5ba74e909744 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:16.765318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47cf75-c882-420e-aae2-5ba74e909744 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:16.765318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47cf75-c882-420e-aae2-5ba74e909744 None admin] RBAC: Adding query filter params (name=network) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:16.765318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47cf75-c882-420e-aae2-5ba74e909744 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:16.765318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47cf75-c882-420e-aae2-5ba74e909744 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:16.769758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 360/718] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:16.775834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908297c7-d876-4939-90be-9e837042c26e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:16.889041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908297c7-d876-4939-90be-9e837042c26e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:16.939376 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abef273-453b-4cb8-992e-681ebffbaed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:16.940582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abef273-453b-4cb8-992e-681ebffbaed2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:16.941019 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abef273-453b-4cb8-992e-681ebffbaed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:16.941462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abef273-453b-4cb8-992e-681ebffbaed2 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:16.942300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abef273-453b-4cb8-992e-681ebffbaed2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:16.942764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abef273-453b-4cb8-992e-681ebffbaed2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:16.944299 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abef273-453b-4cb8-992e-681ebffbaed2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:16.953612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 359/719] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:16 2022] GET /identity/v3/services?type=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:17.395924 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d555e70-d1f3-4b64-858c-d740f8caaeb3 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:17.396478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d555e70-d1f3-4b64-858c-d740f8caaeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:17.396880 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d555e70-d1f3-4b64-858c-d740f8caaeb3 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:17.398570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 361/720] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:17.404191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a635098-5381-4d12-b29c-6e2998b170e8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:17.404589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a635098-5381-4d12-b29c-6e2998b170e8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:17.404965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a635098-5381-4d12-b29c-6e2998b170e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:17.425381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1a635098-5381-4d12-b29c-6e2998b170e8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:17.426751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1a635098-5381-4d12-b29c-6e2998b170e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:17.474849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:17.502695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 360/721] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:24:17.536461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d555e70-d1f3-4b64-858c-d740f8caaeb3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:17.677596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d555e70-d1f3-4b64-858c-d740f8caaeb3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:17.723110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed9452-ee70-4912-b8e3-6bfbff2b3113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:17.723894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed9452-ee70-4912-b8e3-6bfbff2b3113 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:17.724102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed9452-ee70-4912-b8e3-6bfbff2b3113 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:17.724320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed9452-ee70-4912-b8e3-6bfbff2b3113 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:17.735616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed9452-ee70-4912-b8e3-6bfbff2b3113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-30ed9452-ee70-4912-b8e3-6bfbff2b3113 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:17.741579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:17.743675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:17.745746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 362/722] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:17.754191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a635098-5381-4d12-b29c-6e2998b170e8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:17.901905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a635098-5381-4d12-b29c-6e2998b170e8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:17.959191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d08cc-a459-470b-b72d-134bd23e18eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:17.960140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d08cc-a459-470b-b72d-134bd23e18eb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:17.960140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d08cc-a459-470b-b72d-134bd23e18eb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:17.960435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d08cc-a459-470b-b72d-134bd23e18eb None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:17.961120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d08cc-a459-470b-b72d-134bd23e18eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:17.961427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d08cc-a459-470b-b72d-134bd23e18eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:17.962805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d08cc-a459-470b-b72d-134bd23e18eb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:17.975904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 361/723] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:17.981015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed9452-ee70-4912-b8e3-6bfbff2b3113 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:18.048658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:18.104436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed9452-ee70-4912-b8e3-6bfbff2b3113 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:18.139922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc7590-eb06-4ed9-b934-401059c3921a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:18.140591 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7590-eb06-4ed9-b934-401059c3921a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:18.140881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7590-eb06-4ed9-b934-401059c3921a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:18.141157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7590-eb06-4ed9-b934-401059c3921a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:18.161476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc7590-eb06-4ed9-b934-401059c3921a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:18.162949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc7590-eb06-4ed9-b934-401059c3921a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-86bc7590-eb06-4ed9-b934-401059c3921a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:18.180705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:18.181687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:18.183207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:18.183207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:18.183207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:18.183207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:18.183207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 363/724] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:18.187348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d08cc-a459-470b-b72d-134bd23e18eb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:18.237995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:18.324669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d08cc-a459-470b-b72d-134bd23e18eb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:18.368881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75523461-ba74-469e-b9c3-9b24be137a7d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:18.369183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75523461-ba74-469e-b9c3-9b24be137a7d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:18.369426 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75523461-ba74-469e-b9c3-9b24be137a7d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:18.371005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 364/725] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:18.372831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c1f70-8009-466e-9fcb-1c8baa936f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:18.373677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c1f70-8009-466e-9fcb-1c8baa936f47 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:18.374179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c1f70-8009-466e-9fcb-1c8baa936f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:18.374454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c1f70-8009-466e-9fcb-1c8baa936f47 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:18.375246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c1f70-8009-466e-9fcb-1c8baa936f47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:18.375554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c1f70-8009-466e-9fcb-1c8baa936f47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:18.376708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c6059-fdab-480b-8cc8-b348f23336bb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:18.376935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c1f70-8009-466e-9fcb-1c8baa936f47 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:18.377976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c6059-fdab-480b-8cc8-b348f23336bb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:18.378575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c6059-fdab-480b-8cc8-b348f23336bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:18.398554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3b8c6059-fdab-480b-8cc8-b348f23336bb None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:18.399889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3b8c6059-fdab-480b-8cc8-b348f23336bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:18.401284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 362/726] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:18.407126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c1f70-8009-466e-9fcb-1c8baa936f47 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:18.481116 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 365/727] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:18.515874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c6059-fdab-480b-8cc8-b348f23336bb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:18.567491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:18.595285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:18.600865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c1f70-8009-466e-9fcb-1c8baa936f47 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:18.652733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e73163-b060-494f-8f09-3c45cd7cd9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:18.653399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e73163-b060-494f-8f09-3c45cd7cd9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:18.653690 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e73163-b060-494f-8f09-3c45cd7cd9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:18.653976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e73163-b060-494f-8f09-3c45cd7cd9e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:18.662648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e73163-b060-494f-8f09-3c45cd7cd9e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:18.664123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e73163-b060-494f-8f09-3c45cd7cd9e2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-40e73163-b060-494f-8f09-3c45cd7cd9e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:18.676952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:18.677926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:18.679332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:18.679332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:18.679332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:18.679332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:18.679332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:18.679332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:18.679332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:18.679332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:18.679332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:18.679332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:18.679332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 363/728] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Aug 4 07:24:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:18.684048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e73163-b060-494f-8f09-3c45cd7cd9e2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:18.707870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c6059-fdab-480b-8cc8-b348f23336bb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:18.785603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b273a92-e1dc-4829-809c-c4314e47e85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:18.786240 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b273a92-e1dc-4829-809c-c4314e47e85c None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:18.786457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b273a92-e1dc-4829-809c-c4314e47e85c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:18.786700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b273a92-e1dc-4829-809c-c4314e47e85c None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:18.787433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b273a92-e1dc-4829-809c-c4314e47e85c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:18.789022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b273a92-e1dc-4829-809c-c4314e47e85c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:18.797370 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e73163-b060-494f-8f09-3c45cd7cd9e2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:18.835500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054c880-2e56-498d-825b-d0d7e4120a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:18.836157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054c880-2e56-498d-825b-d0d7e4120a90 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:18.836443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054c880-2e56-498d-825b-d0d7e4120a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:18.836723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054c880-2e56-498d-825b-d0d7e4120a90 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:18.837531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054c880-2e56-498d-825b-d0d7e4120a90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:18.837843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054c880-2e56-498d-825b-d0d7e4120a90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:18.839353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054c880-2e56-498d-825b-d0d7e4120a90 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:18.858948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 364/729] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Aug 4 07:24:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:18.864724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054c880-2e56-498d-825b-d0d7e4120a90 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:18.884256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:18.914046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 366/730] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:24:18 2022] POST /identity/v3/services => generated 251 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:19.031305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054c880-2e56-498d-825b-d0d7e4120a90 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:19.084555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3ae2a-801a-42a8-a1d3-3441ee00de75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:19.085265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3ae2a-801a-42a8-a1d3-3441ee00de75 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.085557 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3ae2a-801a-42a8-a1d3-3441ee00de75 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.085887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3ae2a-801a-42a8-a1d3-3441ee00de75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.099053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3ae2a-801a-42a8-a1d3-3441ee00de75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:19.100571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3ae2a-801a-42a8-a1d3-3441ee00de75 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-29f3ae2a-801a-42a8-a1d3-3441ee00de75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:19.116775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:19.118927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:19.121107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:19.121107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:19.121107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:19.121107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:19.121107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 365/731] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:19.129247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b273a92-e1dc-4829-809c-c4314e47e85c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:19.228235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:19.238041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b273a92-e1dc-4829-809c-c4314e47e85c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:19.268861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969d58d-3b2a-4a89-95a8-fe44f07a4d5e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.269209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969d58d-3b2a-4a89-95a8-fe44f07a4d5e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.269505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969d58d-3b2a-4a89-95a8-fe44f07a4d5e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.271184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 366/732] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:24:19.276337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2caa9-47d5-4faf-8149-c001ad2a9b5c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.277010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156be2e-395a-4f15-890d-b8d308f18050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:19.277236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2caa9-47d5-4faf-8149-c001ad2a9b5c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.277236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2caa9-47d5-4faf-8149-c001ad2a9b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.277524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156be2e-395a-4f15-890d-b8d308f18050 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.277524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156be2e-395a-4f15-890d-b8d308f18050 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.277813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156be2e-395a-4f15-890d-b8d308f18050 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.278334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156be2e-395a-4f15-890d-b8d308f18050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:19.278643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156be2e-395a-4f15-890d-b8d308f18050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:19.280041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156be2e-395a-4f15-890d-b8d308f18050 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:19.294577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 367/733] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:19.297052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-25b2caa9-47d5-4faf-8149-c001ad2a9b5c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:19.300135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-25b2caa9-47d5-4faf-8149-c001ad2a9b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:19.302371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156be2e-395a-4f15-890d-b8d308f18050 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:19.375711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:19.387436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 367/734] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 07:24:19.414160 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2caa9-47d5-4faf-8149-c001ad2a9b5c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:19.424261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156be2e-395a-4f15-890d-b8d308f18050 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:19.465237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b91d6-3141-4daa-ae9c-3311cbd5176a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:19.466229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b91d6-3141-4daa-ae9c-3311cbd5176a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.466935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b91d6-3141-4daa-ae9c-3311cbd5176a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.467418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b91d6-3141-4daa-ae9c-3311cbd5176a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.475519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b91d6-3141-4daa-ae9c-3311cbd5176a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:19.477280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b91d6-3141-4daa-ae9c-3311cbd5176a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-160b91d6-3141-4daa-ae9c-3311cbd5176a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:19.489555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:19.490481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:19.491453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:19.491453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:19.491453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:19.491453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:19.491453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:19.491453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:19.491453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:19.491453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:19.493871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 368/735] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:19.495359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f1cc8-bfed-4635-a249-7737099eec70 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.495854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f1cc8-bfed-4635-a249-7737099eec70 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.496369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f1cc8-bfed-4635-a249-7737099eec70 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.501860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 369/736] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:19.502988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f1cc8-bfed-4635-a249-7737099eec70 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:19.519675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:19.527486 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2caa9-47d5-4faf-8149-c001ad2a9b5c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:19.564165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482f10d-f04a-4266-8406-3a0ded916c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:19.565325 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482f10d-f04a-4266-8406-3a0ded916c92 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.565606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482f10d-f04a-4266-8406-3a0ded916c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.565879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482f10d-f04a-4266-8406-3a0ded916c92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.578261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482f10d-f04a-4266-8406-3a0ded916c92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f482f10d-f04a-4266-8406-3a0ded916c92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:19.583729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:19.584727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:19.585589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 368/737] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:19.586306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce2fa1-0455-4d29-a32a-0f1d62e5a989 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.586306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce2fa1-0455-4d29-a32a-0f1d62e5a989 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.586929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce2fa1-0455-4d29-a32a-0f1d62e5a989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.604329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-83ce2fa1-0455-4d29-a32a-0f1d62e5a989 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:19.605501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-83ce2fa1-0455-4d29-a32a-0f1d62e5a989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:19.621139 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f1cc8-bfed-4635-a249-7737099eec70 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:19.654880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:19.657636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2664a1-fed9-4638-b9c5-6a383c62141f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:19.658254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2664a1-fed9-4638-b9c5-6a383c62141f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.658607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2664a1-fed9-4638-b9c5-6a383c62141f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.658889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2664a1-fed9-4638-b9c5-6a383c62141f None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.659642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2664a1-fed9-4638-b9c5-6a383c62141f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:19.659954 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2664a1-fed9-4638-b9c5-6a383c62141f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:19.661406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2664a1-fed9-4638-b9c5-6a383c62141f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:19.680023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 369/738] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:19.680212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 370/739] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:19.680212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2664a1-fed9-4638-b9c5-6a383c62141f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:19.683321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7034e2b-54a5-4e92-be92-ca94e9cba457 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.683321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7034e2b-54a5-4e92-be92-ca94e9cba457 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.683321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7034e2b-54a5-4e92-be92-ca94e9cba457 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.683321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 370/740] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:19.683321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7034e2b-54a5-4e92-be92-ca94e9cba457 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:19.757591 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:19.788422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2664a1-fed9-4638-b9c5-6a383c62141f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:19.799972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7034e2b-54a5-4e92-be92-ca94e9cba457 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:19.824565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc779d-c893-4e86-bec1-b2db94881557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:19.825187 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc779d-c893-4e86-bec1-b2db94881557 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.825474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc779d-c893-4e86-bec1-b2db94881557 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.825754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc779d-c893-4e86-bec1-b2db94881557 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.826466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc779d-c893-4e86-bec1-b2db94881557 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:19.826775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc779d-c893-4e86-bec1-b2db94881557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:19.828212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc779d-c893-4e86-bec1-b2db94881557 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:19.835403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a895daf-b045-4112-b6eb-139b79e94c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:19.836104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a895daf-b045-4112-b6eb-139b79e94c05 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.836391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a895daf-b045-4112-b6eb-139b79e94c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.836674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a895daf-b045-4112-b6eb-139b79e94c05 None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/4eedd711c6114de1bfcdf692ce058794/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.841410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 371/741] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:19.841410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19f384-efa4-45e3-a10f-4fc976aa5ad4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:19.841410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19f384-efa4-45e3-a10f-4fc976aa5ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:19.841410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19f384-efa4-45e3-a10f-4fc976aa5ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:19.842817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a895daf-b045-4112-b6eb-139b79e94c05 None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=4eedd711c6114de1bfcdf692ce058794, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:19.844308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a895daf-b045-4112-b6eb-139b79e94c05 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:19.856926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cf19f384-efa4-45e3-a10f-4fc976aa5ad4 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:19.858341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cf19f384-efa4-45e3-a10f-4fc976aa5ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:19.917312 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 371/742] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:24:19 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/4eedd711c6114de1bfcdf692ce058794/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:24:19.918547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a895daf-b045-4112-b6eb-139b79e94c05 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:19.936151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 372/743] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 07:24:19.937532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19f384-efa4-45e3-a10f-4fc976aa5ad4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:19.995234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:20.044145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:20.061931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a895daf-b045-4112-b6eb-139b79e94c05 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:20.088251 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19f384-efa4-45e3-a10f-4fc976aa5ad4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:20.103350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281edb82-0710-4f9d-add6-735bf520a09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:20.104265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281edb82-0710-4f9d-add6-735bf520a09a None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:20.104692 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281edb82-0710-4f9d-add6-735bf520a09a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:20.105129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281edb82-0710-4f9d-add6-735bf520a09a None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:20.105973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281edb82-0710-4f9d-add6-735bf520a09a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:20.107749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281edb82-0710-4f9d-add6-735bf520a09a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:20.130033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9e5e7-7686-410c-94c9-400bc03fc733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:20.130684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9e5e7-7686-410c-94c9-400bc03fc733 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:20.130967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9e5e7-7686-410c-94c9-400bc03fc733 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:20.131243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9e5e7-7686-410c-94c9-400bc03fc733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:20.143374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9e5e7-7686-410c-94c9-400bc03fc733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:20.144858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9e5e7-7686-410c-94c9-400bc03fc733 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cdb9e5e7-7686-410c-94c9-400bc03fc733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:20.159687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:20.165100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:20.165976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:20.165976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:20.165976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:20.165976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:20.165976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 373/744] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:20.165976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9e5e7-7686-410c-94c9-400bc03fc733 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:20.192365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 372/745] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:24:19 2022] POST /identity/v3/services => generated 240 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:20.193242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281edb82-0710-4f9d-add6-735bf520a09a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:20.337821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9e5e7-7686-410c-94c9-400bc03fc733 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:20.374978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:20.393476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:20.398944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281edb82-0710-4f9d-add6-735bf520a09a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:20.406069 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b0f4b-e3e7-4bc5-b19c-055c079da306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:20.406721 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b0f4b-e3e7-4bc5-b19c-055c079da306 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:20.407000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b0f4b-e3e7-4bc5-b19c-055c079da306 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:20.407270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b0f4b-e3e7-4bc5-b19c-055c079da306 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:20.428441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b0f4b-e3e7-4bc5-b19c-055c079da306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fd7b0f4b-e3e7-4bc5-b19c-055c079da306 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:20.434362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:20.435293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:20.436178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:20.439085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 374/746] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:20.445112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b0f4b-e3e7-4bc5-b19c-055c079da306 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:20.445370 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa194406-c495-4d47-9858-46ba2b3310e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:20.446070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa194406-c495-4d47-9858-46ba2b3310e5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:20.446365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa194406-c495-4d47-9858-46ba2b3310e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:20.446636 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa194406-c495-4d47-9858-46ba2b3310e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:20.447347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa194406-c495-4d47-9858-46ba2b3310e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:20.447654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa194406-c495-4d47-9858-46ba2b3310e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:20.449292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa194406-c495-4d47-9858-46ba2b3310e5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:20.471828 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 373/747] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:20.478735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa194406-c495-4d47-9858-46ba2b3310e5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:20.568896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b0f4b-e3e7-4bc5-b19c-055c079da306 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:20.629994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa194406-c495-4d47-9858-46ba2b3310e5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:20.637152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48652acb-c3cd-4fe2-8678-08fc4025cd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:20.638021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48652acb-c3cd-4fe2-8678-08fc4025cd8b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:20.638475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48652acb-c3cd-4fe2-8678-08fc4025cd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:20.638908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48652acb-c3cd-4fe2-8678-08fc4025cd8b None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:20.639774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48652acb-c3cd-4fe2-8678-08fc4025cd8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:20.640252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48652acb-c3cd-4fe2-8678-08fc4025cd8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:20.641788 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48652acb-c3cd-4fe2-8678-08fc4025cd8b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:20.659383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 375/748] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:20.663085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48652acb-c3cd-4fe2-8678-08fc4025cd8b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:20.699864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf317a-d457-4d4f-b5b0-7abee0a6e4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:20.700771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf317a-d457-4d4f-b5b0-7abee0a6e4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:20.701273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf317a-d457-4d4f-b5b0-7abee0a6e4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:20.701817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf317a-d457-4d4f-b5b0-7abee0a6e4f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:20.722233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:20.722778 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf317a-d457-4d4f-b5b0-7abee0a6e4f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:20.726517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf317a-d457-4d4f-b5b0-7abee0a6e4f9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8caf317a-d457-4d4f-b5b0-7abee0a6e4f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:20.744880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:20.745898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:20.746865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:20.746865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:20.746865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:20.746865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:20.746865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:20.746865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:20.746865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:20.746865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:20.746865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:20.746865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:20.748645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 374/749] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Aug 4 07:24:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:20.752763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf317a-d457-4d4f-b5b0-7abee0a6e4f9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:20.793993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:20.837427 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48652acb-c3cd-4fe2-8678-08fc4025cd8b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:20.877990 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce93c-3995-404f-8419-f95fd43bc925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:20.878645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce93c-3995-404f-8419-f95fd43bc925 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:20.878935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce93c-3995-404f-8419-f95fd43bc925 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:20.879216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce93c-3995-404f-8419-f95fd43bc925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:20.891251 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ce93c-3995-404f-8419-f95fd43bc925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:20.893388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ce93c-3995-404f-8419-f95fd43bc925 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-006ce93c-3995-404f-8419-f95fd43bc925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:20.911318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:20.912368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:20.913259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:20.913259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:20.913259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:20.913259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:20.913479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 376/750] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:20.918727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce93c-3995-404f-8419-f95fd43bc925 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:20.947289 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf317a-d457-4d4f-b5b0-7abee0a6e4f9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:21.000158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43976721-aa4b-49f3-bbd6-075a762968b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:21.000840 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43976721-aa4b-49f3-bbd6-075a762968b9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:21.001124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43976721-aa4b-49f3-bbd6-075a762968b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:21.001397 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43976721-aa4b-49f3-bbd6-075a762968b9 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:21.002229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43976721-aa4b-49f3-bbd6-075a762968b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:21.002769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43976721-aa4b-49f3-bbd6-075a762968b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:21.004338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43976721-aa4b-49f3-bbd6-075a762968b9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:21.023832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 375/751] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Aug 4 07:24:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:21.030641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43976721-aa4b-49f3-bbd6-075a762968b9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:21.049109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:21.059859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce93c-3995-404f-8419-f95fd43bc925 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:21.115699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a642b-007f-4437-b4e7-e7650d8ab3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:21.116405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a642b-007f-4437-b4e7-e7650d8ab3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:21.116687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a642b-007f-4437-b4e7-e7650d8ab3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:21.116961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a642b-007f-4437-b4e7-e7650d8ab3f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:21.117660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a642b-007f-4437-b4e7-e7650d8ab3f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:21.117966 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a642b-007f-4437-b4e7-e7650d8ab3f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:21.119346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a642b-007f-4437-b4e7-e7650d8ab3f7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:21.144737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 377/752] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:21.156300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a642b-007f-4437-b4e7-e7650d8ab3f7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:21.160128 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:21.195823 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43976721-aa4b-49f3-bbd6-075a762968b9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:21.259220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd39b55-d37a-4ea6-8fe8-7015b8f6a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:21.259770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd39b55-d37a-4ea6-8fe8-7015b8f6a855 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:21.260137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd39b55-d37a-4ea6-8fe8-7015b8f6a855 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:21.260423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd39b55-d37a-4ea6-8fe8-7015b8f6a855 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:21.276200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a642b-007f-4437-b4e7-e7650d8ab3f7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:21.320551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21510eed-f9ff-4c40-b797-89299a7e87d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:21.321223 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21510eed-f9ff-4c40-b797-89299a7e87d6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:21.321512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21510eed-f9ff-4c40-b797-89299a7e87d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:21.321789 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21510eed-f9ff-4c40-b797-89299a7e87d6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:21.341270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21510eed-f9ff-4c40-b797-89299a7e87d6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:21.342761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21510eed-f9ff-4c40-b797-89299a7e87d6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:21.351742 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd39b55-d37a-4ea6-8fe8-7015b8f6a855 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:21.353261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd39b55-d37a-4ea6-8fe8-7015b8f6a855 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-21510eed-f9ff-4c40-b797-89299a7e87d6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:21.356274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:21.357180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:21.358745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:21.358745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:21.358745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:21.358745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:21.358745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:21.358745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:21.358745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:21.358745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:21.358745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:21.358745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:21.358745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 378/753] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:21.359338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164cf5e-ae12-41bb-9e06-ac76d2689656 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:21.359641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164cf5e-ae12-41bb-9e06-ac76d2689656 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:21.360736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164cf5e-ae12-41bb-9e06-ac76d2689656 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:21.362231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 379/754] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:21.366490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164cf5e-ae12-41bb-9e06-ac76d2689656 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:21.384335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:21.434914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:21.459263 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 376/755] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:24:21.460796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed92c7-e520-4e11-82af-30b525fdb036 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:21.461271 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed92c7-e520-4e11-82af-30b525fdb036 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:21.461727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed92c7-e520-4e11-82af-30b525fdb036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:21.474290 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164cf5e-ae12-41bb-9e06-ac76d2689656 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:21.480905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-64ed92c7-e520-4e11-82af-30b525fdb036 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:21.482576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-64ed92c7-e520-4e11-82af-30b525fdb036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:21.510503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3270620-30f5-4099-b020-2edf18776fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:21.511130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3270620-30f5-4099-b020-2edf18776fee None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:21.511790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3270620-30f5-4099-b020-2edf18776fee None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:21.511912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3270620-30f5-4099-b020-2edf18776fee None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:21.512755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3270620-30f5-4099-b020-2edf18776fee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:21.513131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3270620-30f5-4099-b020-2edf18776fee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:21.514597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3270620-30f5-4099-b020-2edf18776fee None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:21.532422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 380/756] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Aug 4 07:24:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:21.533590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3270620-30f5-4099-b020-2edf18776fee None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:21.554673 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 377/757] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:21.555749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed92c7-e520-4e11-82af-30b525fdb036 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:21.624534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:21.661208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3270620-30f5-4099-b020-2edf18776fee None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:21.678969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed92c7-e520-4e11-82af-30b525fdb036 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:21.692727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:21.699158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd2481-570f-4f55-a167-a0eb83c8a004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:21.700242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd2481-570f-4f55-a167-a0eb83c8a004 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:21.700719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd2481-570f-4f55-a167-a0eb83c8a004 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:21.701142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd2481-570f-4f55-a167-a0eb83c8a004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:21.713564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd2481-570f-4f55-a167-a0eb83c8a004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:21.715559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd2481-570f-4f55-a167-a0eb83c8a004 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:21.722474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f73cf-fe03-44dd-98a6-f6e8da7c7049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:21.723179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f73cf-fe03-44dd-98a6-f6e8da7c7049 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:21.723531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f73cf-fe03-44dd-98a6-f6e8da7c7049 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:21.723862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f73cf-fe03-44dd-98a6-f6e8da7c7049 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-43fd2481-570f-4f55-a167-a0eb83c8a004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:21.733986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:21.735090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:21.736185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:21.736185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:21.736185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:21.736185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:21.738902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 381/758] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:21.740058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd2481-570f-4f55-a167-a0eb83c8a004 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:21.822092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f73cf-fe03-44dd-98a6-f6e8da7c7049 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:21.823469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f73cf-fe03-44dd-98a6-f6e8da7c7049 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:21.851653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd2481-570f-4f55-a167-a0eb83c8a004 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:21.873534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:21.889408 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cd3a7-9f98-4999-865c-cbfb5d0d246f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:21.890195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cd3a7-9f98-4999-865c-cbfb5d0d246f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:21.890566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cd3a7-9f98-4999-865c-cbfb5d0d246f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:21.890949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cd3a7-9f98-4999-865c-cbfb5d0d246f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:21.891844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52cd3a7-9f98-4999-865c-cbfb5d0d246f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:21.894322 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52cd3a7-9f98-4999-865c-cbfb5d0d246f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d52cd3a7-9f98-4999-865c-cbfb5d0d246f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:21.909463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:21.910418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 07:24:21.911692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:21.915921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 382/759] 10.222.0.64 () {56 vars in 1129 bytes} [Thu Aug 4 07:24:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:21.916791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cd3a7-9f98-4999-865c-cbfb5d0d246f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:21.941256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 378/760] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:24:21.942637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f73cf-fe03-44dd-98a6-f6e8da7c7049 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:21.957203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:22.042618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cd3a7-9f98-4999-865c-cbfb5d0d246f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:22.062786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f73cf-fe03-44dd-98a6-f6e8da7c7049 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:22.085905 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bc1b1-2f29-4839-8ae8-890fd838b72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:22.087414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc1b1-2f29-4839-8ae8-890fd838b72f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.088061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc1b1-2f29-4839-8ae8-890fd838b72f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.088680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc1b1-2f29-4839-8ae8-890fd838b72f None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.089853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bc1b1-2f29-4839-8ae8-890fd838b72f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:22.090817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bc1b1-2f29-4839-8ae8-890fd838b72f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:22.093022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bc1b1-2f29-4839-8ae8-890fd838b72f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:22.103038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a9511-618e-4622-99c9-e0a68e165e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:22.103785 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9511-618e-4622-99c9-e0a68e165e36 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.104100 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9511-618e-4622-99c9-e0a68e165e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.104385 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9511-618e-4622-99c9-e0a68e165e36 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.105522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a9511-618e-4622-99c9-e0a68e165e36 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:22.105848 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a9511-618e-4622-99c9-e0a68e165e36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:22.107811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a9511-618e-4622-99c9-e0a68e165e36 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:22.110202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 383/761] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:22.111678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bc1b1-2f29-4839-8ae8-890fd838b72f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:22.118479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 379/762] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Aug 4 07:24:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:22.119359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a9511-618e-4622-99c9-e0a68e165e36 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:22.165494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:22.202712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:22.223086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bc1b1-2f29-4839-8ae8-890fd838b72f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:22.238995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a9511-618e-4622-99c9-e0a68e165e36 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:22.258785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8455d-980e-402b-96b4-076a6c348b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:22.259477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8455d-980e-402b-96b4-076a6c348b61 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.259768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8455d-980e-402b-96b4-076a6c348b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.260131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8455d-980e-402b-96b4-076a6c348b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.272906 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8455d-980e-402b-96b4-076a6c348b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:22.274368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8455d-980e-402b-96b4-076a6c348b61 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:22.281817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980cb25-08af-4bf9-a56f-4f9ca8d84d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:22.282455 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980cb25-08af-4bf9-a56f-4f9ca8d84d24 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.282742 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980cb25-08af-4bf9-a56f-4f9ca8d84d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.283017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980cb25-08af-4bf9-a56f-4f9ca8d84d24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b5b8455d-980e-402b-96b4-076a6c348b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:22.290135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:22.291045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:22.291945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:22.291945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:22.291945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:22.291945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:22.292239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980cb25-08af-4bf9-a56f-4f9ca8d84d24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:22.292239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980cb25-08af-4bf9-a56f-4f9ca8d84d24 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:22.298179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 384/763] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:22.298179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8455d-980e-402b-96b4-076a6c348b61 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2980cb25-08af-4bf9-a56f-4f9ca8d84d24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:22.305292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:22.306176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:22.307086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:22.307086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:22.307086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:22.307086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:22.307086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:22.307086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:22.307086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:22.307086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:22.308133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 380/764] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:22.309595 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d91e-d658-4db5-9840-299da7683efd None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.310096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d91e-d658-4db5-9840-299da7683efd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.310674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d91e-d658-4db5-9840-299da7683efd None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.312432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 381/765] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:22.313387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d91e-d658-4db5-9840-299da7683efd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:22.397119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:22.403500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8455d-980e-402b-96b4-076a6c348b61 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:22.422877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d91e-d658-4db5-9840-299da7683efd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:22.438827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a46df-5bd4-4bdb-8346-23450a4eb74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:22.439446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a46df-5bd4-4bdb-8346-23450a4eb74f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.439763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a46df-5bd4-4bdb-8346-23450a4eb74f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.440604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a46df-5bd4-4bdb-8346-23450a4eb74f None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.441446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a46df-5bd4-4bdb-8346-23450a4eb74f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:22.441750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a46df-5bd4-4bdb-8346-23450a4eb74f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:22.443256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a46df-5bd4-4bdb-8346-23450a4eb74f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:22.449111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 385/766] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Aug 4 07:24:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:22.449887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a46df-5bd4-4bdb-8346-23450a4eb74f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:22.459748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:22.465300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c243442-de1d-4250-ba56-7ad7c4d4ee3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:22.466088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c243442-de1d-4250-ba56-7ad7c4d4ee3d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.466561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c243442-de1d-4250-ba56-7ad7c4d4ee3d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.473349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c243442-de1d-4250-ba56-7ad7c4d4ee3d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.473349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c243442-de1d-4250-ba56-7ad7c4d4ee3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:22.473349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c243442-de1d-4250-ba56-7ad7c4d4ee3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:22.473349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c243442-de1d-4250-ba56-7ad7c4d4ee3d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:22.482640 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 382/767] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:22.484139 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896233a8-9a28-4e07-bc75-51cb94af6982 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.484465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896233a8-9a28-4e07-bc75-51cb94af6982 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.484697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896233a8-9a28-4e07-bc75-51cb94af6982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.502853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-896233a8-9a28-4e07-bc75-51cb94af6982 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:22.504158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-896233a8-9a28-4e07-bc75-51cb94af6982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:22.557524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a46df-5bd4-4bdb-8346-23450a4eb74f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:22.584260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 383/768] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:22.585053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896233a8-9a28-4e07-bc75-51cb94af6982 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:22.591219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaed61d-ee32-4730-8fa4-da062d7261cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:22.591830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed61d-ee32-4730-8fa4-da062d7261cc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.592152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed61d-ee32-4730-8fa4-da062d7261cc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.592429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed61d-ee32-4730-8fa4-da062d7261cc None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.593185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaed61d-ee32-4730-8fa4-da062d7261cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:22.593480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaed61d-ee32-4730-8fa4-da062d7261cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:22.594914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaed61d-ee32-4730-8fa4-da062d7261cc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:22.609837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 386/769] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:22.610648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaed61d-ee32-4730-8fa4-da062d7261cc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:22.633281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:22.701657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896233a8-9a28-4e07-bc75-51cb94af6982 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:22.711135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaed61d-ee32-4730-8fa4-da062d7261cc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:22.724731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:22.736205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97984ee7-c3c5-4689-a041-71a87a7357c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:22.736890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97984ee7-c3c5-4689-a041-71a87a7357c7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.747691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97984ee7-c3c5-4689-a041-71a87a7357c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.747691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97984ee7-c3c5-4689-a041-71a87a7357c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.747691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97984ee7-c3c5-4689-a041-71a87a7357c7 None admin] RBAC: Adding query filter params (interface=public, service_id=a120b287c9e94db6b4d96ab7a801c9ce, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:22.747691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97984ee7-c3c5-4689-a041-71a87a7357c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:22.747691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97984ee7-c3c5-4689-a041-71a87a7357c7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:22.748137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfed00-dcd2-4fef-a3f5-f9bd011a9a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:22.748137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfed00-dcd2-4fef-a3f5-f9bd011a9a37 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.748137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfed00-dcd2-4fef-a3f5-f9bd011a9a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.748137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfed00-dcd2-4fef-a3f5-f9bd011a9a37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.748429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 384/770] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Aug 4 07:24:22 2022] GET /identity/v3/endpoints?service_id=a120b287c9e94db6b4d96ab7a801c9ce&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:22.748429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97984ee7-c3c5-4689-a041-71a87a7357c7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:22.757384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfed00-dcd2-4fef-a3f5-f9bd011a9a37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:22.758829 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfed00-dcd2-4fef-a3f5-f9bd011a9a37 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-47bfed00-dcd2-4fef-a3f5-f9bd011a9a37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:22.770892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:22.771767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:22.772696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:22.772696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:22.772696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:22.772696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:22.772696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:22.772696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:22.772696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:22.772696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:22.772696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 387/771] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:22.773180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfed00-dcd2-4fef-a3f5-f9bd011a9a37 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:22.849513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97984ee7-c3c5-4689-a041-71a87a7357c7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:22.851755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:22.871129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfed00-dcd2-4fef-a3f5-f9bd011a9a37 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:22.883572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e28be-f52d-4497-a256-b105082892d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:22.884225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e28be-f52d-4497-a256-b105082892d8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.884506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e28be-f52d-4497-a256-b105082892d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.884781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e28be-f52d-4497-a256-b105082892d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.895098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e28be-f52d-4497-a256-b105082892d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e91e28be-f52d-4497-a256-b105082892d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:22.900359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:22.901389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:22.902233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 385/772] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:22.902916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc07a1-c5e4-41e5-a874-679a954b5a4f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.902983 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc07a1-c5e4-41e5-a874-679a954b5a4f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.903231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc07a1-c5e4-41e5-a874-679a954b5a4f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.904496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 386/773] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:22.905170 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc07a1-c5e4-41e5-a874-679a954b5a4f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:22.911018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6714e0e-47de-4afc-9966-c48bdc037282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:22.911629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6714e0e-47de-4afc-9966-c48bdc037282 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:22.911909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6714e0e-47de-4afc-9966-c48bdc037282 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:22.912212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6714e0e-47de-4afc-9966-c48bdc037282 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:22.912999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6714e0e-47de-4afc-9966-c48bdc037282 None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=0c30c2fd1c4648308fac6606526451ec) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:22.913307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6714e0e-47de-4afc-9966-c48bdc037282 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:22.914737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6714e0e-47de-4afc-9966-c48bdc037282 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:22.920521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 388/774] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:22 2022] GET /identity/v3/role_assignments?user.id=0c30c2fd1c4648308fac6606526451ec&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:22.921275 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6714e0e-47de-4afc-9966-c48bdc037282 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:22.978963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:23.042673 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc07a1-c5e4-41e5-a874-679a954b5a4f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:23.043409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6714e0e-47de-4afc-9966-c48bdc037282 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:23.083658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b0cdf-93a8-4b47-bfde-d694c8d2cead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:23.084314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0cdf-93a8-4b47-bfde-d694c8d2cead None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:23.088810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0cdf-93a8-4b47-bfde-d694c8d2cead None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:23.088935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0ce1b-550b-40e2-9704-8a9522316c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:23.089055 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0cdf-93a8-4b47-bfde-d694c8d2cead None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:23.089132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0ce1b-550b-40e2-9704-8a9522316c2f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:23.089201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b0cdf-93a8-4b47-bfde-d694c8d2cead None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:23.089268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0ce1b-550b-40e2-9704-8a9522316c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:23.089344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b0cdf-93a8-4b47-bfde-d694c8d2cead None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:23.089424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0ce1b-550b-40e2-9704-8a9522316c2f None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:23.089424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0ce1b-550b-40e2-9704-8a9522316c2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:23.089548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b0cdf-93a8-4b47-bfde-d694c8d2cead None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:23.089614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0ce1b-550b-40e2-9704-8a9522316c2f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:23.093259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 387/775] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Aug 4 07:24:22 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:23.094048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0ce1b-550b-40e2-9704-8a9522316c2f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:23.107106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 389/776] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:23.109620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a7b88-ccd1-4e06-8b9a-b558fd4bd0f9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:23.110116 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a7b88-ccd1-4e06-8b9a-b558fd4bd0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:23.110572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a7b88-ccd1-4e06-8b9a-b558fd4bd0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:23.132385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:23.138093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-772a7b88-ccd1-4e06-8b9a-b558fd4bd0f9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:23.139528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-772a7b88-ccd1-4e06-8b9a-b558fd4bd0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:23.213126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0ce1b-550b-40e2-9704-8a9522316c2f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:23.226865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:23.239832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 390/777] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:23.240749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a7b88-ccd1-4e06-8b9a-b558fd4bd0f9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:23.266229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be986b-f7ce-4552-8203-ac0fb7c4ce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:23.268464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be986b-f7ce-4552-8203-ac0fb7c4ce91 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:23.269090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be986b-f7ce-4552-8203-ac0fb7c4ce91 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:23.269536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be986b-f7ce-4552-8203-ac0fb7c4ce91 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:23.270291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be986b-f7ce-4552-8203-ac0fb7c4ce91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:23.270610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be986b-f7ce-4552-8203-ac0fb7c4ce91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:23.272242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be986b-f7ce-4552-8203-ac0fb7c4ce91 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:23.282051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 388/778] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:23.282767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be986b-f7ce-4552-8203-ac0fb7c4ce91 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:23.364394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a7b88-ccd1-4e06-8b9a-b558fd4bd0f9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:23.404600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f8202-55f9-42aa-bd29-f833d611163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:23.405226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f8202-55f9-42aa-bd29-f833d611163e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:23.405511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f8202-55f9-42aa-bd29-f833d611163e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:23.405817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f8202-55f9-42aa-bd29-f833d611163e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:23.407111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f8202-55f9-42aa-bd29-f833d611163e None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=9503e2888d5e4831afc0d9d9522c3eea) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:23.407572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f8202-55f9-42aa-bd29-f833d611163e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:23.409252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f8202-55f9-42aa-bd29-f833d611163e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:23.412041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:23.418251 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 391/779] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:23 2022] GET /identity/v3/role_assignments?user.id=9503e2888d5e4831afc0d9d9522c3eea&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:23.419423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f8202-55f9-42aa-bd29-f833d611163e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:23.428692 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be986b-f7ce-4552-8203-ac0fb7c4ce91 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:23.469522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cc81d-51b2-4fe2-8bd8-8ba3661dcb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:23.470546 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cc81d-51b2-4fe2-8bd8-8ba3661dcb45 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:23.471571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cc81d-51b2-4fe2-8bd8-8ba3661dcb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:23.472154 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cc81d-51b2-4fe2-8bd8-8ba3661dcb45 None admin] PATH_INFO: `/v3/services/network` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:23.473071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cc81d-51b2-4fe2-8bd8-8ba3661dcb45 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:23.474704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cc81d-51b2-4fe2-8bd8-8ba3661dcb45 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-bb7cc81d-51b2-4fe2-8bd8-8ba3661dcb45 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:23.485288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:23.486352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 07:24:23.487795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:23.490709 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 389/780] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:23 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:23.500485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cc81d-51b2-4fe2-8bd8-8ba3661dcb45 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:23.563213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f8202-55f9-42aa-bd29-f833d611163e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:23.586834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:23.619161 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663dca7-4170-4ef2-963e-52ee7a0dacc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:23.619819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663dca7-4170-4ef2-963e-52ee7a0dacc6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:23.620144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663dca7-4170-4ef2-963e-52ee7a0dacc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:23.620425 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663dca7-4170-4ef2-963e-52ee7a0dacc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:23.636253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663dca7-4170-4ef2-963e-52ee7a0dacc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:23.637282 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663dca7-4170-4ef2-963e-52ee7a0dacc6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0663dca7-4170-4ef2-963e-52ee7a0dacc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:23.654693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:23.655636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:23.656994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:23.656994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:23.656994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:23.656994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:23.656994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 392/781] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:23.662263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663dca7-4170-4ef2-963e-52ee7a0dacc6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:23.691379 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cc81d-51b2-4fe2-8bd8-8ba3661dcb45 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:23.759171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af80b1-aee3-46f9-a931-1ef8cea9d193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:23.759717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af80b1-aee3-46f9-a931-1ef8cea9d193 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:23.760019 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af80b1-aee3-46f9-a931-1ef8cea9d193 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:23.760310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af80b1-aee3-46f9-a931-1ef8cea9d193 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:23.761003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af80b1-aee3-46f9-a931-1ef8cea9d193 None admin] RBAC: Adding query filter params (name=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:23.761314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af80b1-aee3-46f9-a931-1ef8cea9d193 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:23.762682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af80b1-aee3-46f9-a931-1ef8cea9d193 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:23.766232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:23.774837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 390/782] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:23.783501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af80b1-aee3-46f9-a931-1ef8cea9d193 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:23.881958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663dca7-4170-4ef2-963e-52ee7a0dacc6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:23.953791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1d72f-ff5a-4bd4-8b5b-4c89e453d380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:23.954420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1d72f-ff5a-4bd4-8b5b-4c89e453d380 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:23.954705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1d72f-ff5a-4bd4-8b5b-4c89e453d380 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:23.955041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1d72f-ff5a-4bd4-8b5b-4c89e453d380 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:23.955798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1d72f-ff5a-4bd4-8b5b-4c89e453d380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:23.956129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1d72f-ff5a-4bd4-8b5b-4c89e453d380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:23.957526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1d72f-ff5a-4bd4-8b5b-4c89e453d380 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:23.960059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:23.963899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af80b1-aee3-46f9-a931-1ef8cea9d193 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:23.974986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 393/783] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:23.989383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1d72f-ff5a-4bd4-8b5b-4c89e453d380 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:24.016566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bcdb1-3212-4ddd-a5ba-95d63a11c4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:24.017115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bcdb1-3212-4ddd-a5ba-95d63a11c4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:24.017403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bcdb1-3212-4ddd-a5ba-95d63a11c4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:24.017705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bcdb1-3212-4ddd-a5ba-95d63a11c4b8 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:24.018404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bcdb1-3212-4ddd-a5ba-95d63a11c4b8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:24.018713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bcdb1-3212-4ddd-a5ba-95d63a11c4b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:24.020107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bcdb1-3212-4ddd-a5ba-95d63a11c4b8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:24.033215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 391/784] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:24.037009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bcdb1-3212-4ddd-a5ba-95d63a11c4b8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:24.194182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1d72f-ff5a-4bd4-8b5b-4c89e453d380 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:24.216320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:24.233747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bcdb1-3212-4ddd-a5ba-95d63a11c4b8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:24.270797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba55ab-384a-4c39-9bee-a861ee478c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:24.271340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba55ab-384a-4c39-9bee-a861ee478c9c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:24.271628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba55ab-384a-4c39-9bee-a861ee478c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:24.271907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba55ab-384a-4c39-9bee-a861ee478c9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:24.272705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba55ab-384a-4c39-9bee-a861ee478c9c None admin] RBAC: Adding query filter params (interface=public, service_id=ec86be7b38f9467383d4988f6158a1ef, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:24.273048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba55ab-384a-4c39-9bee-a861ee478c9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:24.274419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba55ab-384a-4c39-9bee-a861ee478c9c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:24.280257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 392/785] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Aug 4 07:24:24 2022] GET /identity/v3/endpoints?service_id=ec86be7b38f9467383d4988f6158a1ef&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:24.282221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140bd6e-243c-4d31-9bec-6739a9495df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:24.282874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140bd6e-243c-4d31-9bec-6739a9495df0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:24.283155 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140bd6e-243c-4d31-9bec-6739a9495df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:24.283455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140bd6e-243c-4d31-9bec-6739a9495df0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:24.289016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba55ab-384a-4c39-9bee-a861ee478c9c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:24.292285 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140bd6e-243c-4d31-9bec-6739a9495df0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:24.293834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140bd6e-243c-4d31-9bec-6739a9495df0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1140bd6e-243c-4d31-9bec-6739a9495df0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:24.306693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:24.307725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:24.308708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:24.308708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:24.308708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:24.308708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:24.308708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:24.308708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:24.308708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:24.308708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:24.308708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:24.308708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:24.308708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 394/786] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Aug 4 07:24:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:24.316906 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140bd6e-243c-4d31-9bec-6739a9495df0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:24.336757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:24.415405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba55ab-384a-4c39-9bee-a861ee478c9c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:24.473819 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b2fab-cd06-4bc4-aee0-704aef8037a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:24.474879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b2fab-cd06-4bc4-aee0-704aef8037a1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:24.475500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b2fab-cd06-4bc4-aee0-704aef8037a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:24.476889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b2fab-cd06-4bc4-aee0-704aef8037a1 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:24.478504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b2fab-cd06-4bc4-aee0-704aef8037a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:24.480254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b2fab-cd06-4bc4-aee0-704aef8037a1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:24.493446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 393/787] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Aug 4 07:24:24 2022] GET /identity/v3/services => generated 772 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:24.504676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140bd6e-243c-4d31-9bec-6739a9495df0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:24.562376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252edfc8-d926-462e-bb36-ea31264f92eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:24.563365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252edfc8-d926-462e-bb36-ea31264f92eb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:24.563850 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252edfc8-d926-462e-bb36-ea31264f92eb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:24.564363 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252edfc8-d926-462e-bb36-ea31264f92eb None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:24.565447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252edfc8-d926-462e-bb36-ea31264f92eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:24.565978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252edfc8-d926-462e-bb36-ea31264f92eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:24.567790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252edfc8-d926-462e-bb36-ea31264f92eb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:24.592454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 395/788] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Aug 4 07:24:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:24.598451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b2fab-cd06-4bc4-aee0-704aef8037a1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:24.708900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:24.720589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b2fab-cd06-4bc4-aee0-704aef8037a1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:24.757687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fe870-8422-4285-a3c6-fe77dd8df3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:24.758308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fe870-8422-4285-a3c6-fe77dd8df3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:24.758590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fe870-8422-4285-a3c6-fe77dd8df3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:24.758862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fe870-8422-4285-a3c6-fe77dd8df3c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:24.835622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fe870-8422-4285-a3c6-fe77dd8df3c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:24.837022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fe870-8422-4285-a3c6-fe77dd8df3c9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:24.941671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:24.961968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 394/789] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 07:24:24.969768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252edfc8-d926-462e-bb36-ea31264f92eb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:25.013683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:25.090061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252edfc8-d926-462e-bb36-ea31264f92eb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:25.136279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eeb4f7-ba10-403c-bc5f-938dedfc2ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:25.137304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eeb4f7-ba10-403c-bc5f-938dedfc2ffb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:25.137813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eeb4f7-ba10-403c-bc5f-938dedfc2ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:25.138303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eeb4f7-ba10-403c-bc5f-938dedfc2ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:25.153865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eeb4f7-ba10-403c-bc5f-938dedfc2ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:25.155640 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eeb4f7-ba10-403c-bc5f-938dedfc2ffb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e5eeb4f7-ba10-403c-bc5f-938dedfc2ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:25.171613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:25.172576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:25.173492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:25.173492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:25.173492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:25.173492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:25.176225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 396/790] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:25.184015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fe870-8422-4285-a3c6-fe77dd8df3c9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:25.313409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de9645-6825-4fa3-9101-a4b61755b78c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:25.313805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de9645-6825-4fa3-9101-a4b61755b78c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:25.313986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de9645-6825-4fa3-9101-a4b61755b78c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:25.316287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 397/791] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:25.321492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322c86-b3d8-4570-ae87-9a59fa8a0264 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:25.321778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322c86-b3d8-4570-ae87-9a59fa8a0264 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:25.322102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322c86-b3d8-4570-ae87-9a59fa8a0264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:25.339343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fe870-8422-4285-a3c6-fe77dd8df3c9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:25.362290 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b6322c86-b3d8-4570-ae87-9a59fa8a0264 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:25.363572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b6322c86-b3d8-4570-ae87-9a59fa8a0264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:25.379321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b712f-2e1b-40f1-b652-bb88a99a8796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:25.380230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b712f-2e1b-40f1-b652-bb88a99a8796 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:25.380655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b712f-2e1b-40f1-b652-bb88a99a8796 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:25.381097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b712f-2e1b-40f1-b652-bb88a99a8796 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:25.381935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b712f-2e1b-40f1-b652-bb88a99a8796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:25.382962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b712f-2e1b-40f1-b652-bb88a99a8796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:25.384507 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b712f-2e1b-40f1-b652-bb88a99a8796 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:25.398151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 395/792] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:25.403337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b712f-2e1b-40f1-b652-bb88a99a8796 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:25.504021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 398/793] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:25.512217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:25.526940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6322c86-b3d8-4570-ae87-9a59fa8a0264 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:25.548804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:25.591805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b712f-2e1b-40f1-b652-bb88a99a8796 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:25.638220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919f701-51e2-4f49-80e4-767f21555e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:25.639381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919f701-51e2-4f49-80e4-767f21555e22 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:25.640108 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919f701-51e2-4f49-80e4-767f21555e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:25.640562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919f701-51e2-4f49-80e4-767f21555e22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:25.644300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6322c86-b3d8-4570-ae87-9a59fa8a0264 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:25.650633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919f701-51e2-4f49-80e4-767f21555e22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:25.652853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919f701-51e2-4f49-80e4-767f21555e22 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e919f701-51e2-4f49-80e4-767f21555e22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:25.668826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:25.669753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:25.670627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:25.670627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:25.670627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:25.670627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:25.670627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:25.670627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:25.670627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:25.670627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:25.673395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 396/794] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:25.675780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf12005-8c6c-438b-904f-e1ded0543338 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:25.676364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf12005-8c6c-438b-904f-e1ded0543338 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:25.676810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf12005-8c6c-438b-904f-e1ded0543338 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:25.678958 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 397/795] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:25.679851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf12005-8c6c-438b-904f-e1ded0543338 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:25.687534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19a875-3799-47cb-b0f6-9e88ddb0b5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:25.688354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19a875-3799-47cb-b0f6-9e88ddb0b5ee None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:25.688743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19a875-3799-47cb-b0f6-9e88ddb0b5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:25.689142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19a875-3799-47cb-b0f6-9e88ddb0b5ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:25.700644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19a875-3799-47cb-b0f6-9e88ddb0b5ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7b19a875-3799-47cb-b0f6-9e88ddb0b5ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:25.706623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:25.707766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:25.708826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:25.710676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 399/796] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:25.716545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e805a36-c900-44e4-853a-cc16e3f9b441 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:25.716949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e805a36-c900-44e4-853a-cc16e3f9b441 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:25.717325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e805a36-c900-44e4-853a-cc16e3f9b441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:25.735782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9e805a36-c900-44e4-853a-cc16e3f9b441 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:25.737297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e805a36-c900-44e4-853a-cc16e3f9b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:25.805261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:25.816003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 400/797] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:25.816003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e805a36-c900-44e4-853a-cc16e3f9b441 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:25.832688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf12005-8c6c-438b-904f-e1ded0543338 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:25.846909 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:25.868831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f307b5-3e9f-44a8-9f0b-9b12a6f02ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:25.869457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f307b5-3e9f-44a8-9f0b-9b12a6f02ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:25.869739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f307b5-3e9f-44a8-9f0b-9b12a6f02ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:25.870018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f307b5-3e9f-44a8-9f0b-9b12a6f02ab0 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:25.870808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f307b5-3e9f-44a8-9f0b-9b12a6f02ab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:25.871121 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f307b5-3e9f-44a8-9f0b-9b12a6f02ab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:25.872601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f307b5-3e9f-44a8-9f0b-9b12a6f02ab0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:25.886206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 398/798] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:25.886872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f307b5-3e9f-44a8-9f0b-9b12a6f02ab0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:25.917956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e805a36-c900-44e4-853a-cc16e3f9b441 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:25.958652 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacf4b8-f7f8-4b6b-a69e-a1dfe70bdbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:25.959436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacf4b8-f7f8-4b6b-a69e-a1dfe70bdbfe None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:25.959790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacf4b8-f7f8-4b6b-a69e-a1dfe70bdbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:25.960439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacf4b8-f7f8-4b6b-a69e-a1dfe70bdbfe None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:25.960836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacf4b8-f7f8-4b6b-a69e-a1dfe70bdbfe None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:25.961305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacf4b8-f7f8-4b6b-a69e-a1dfe70bdbfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:25.962694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacf4b8-f7f8-4b6b-a69e-a1dfe70bdbfe None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:25.975921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 401/799] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:25.975921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe935a-b213-4ffe-89a9-03fb653041d1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:25.975921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe935a-b213-4ffe-89a9-03fb653041d1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:25.975921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe935a-b213-4ffe-89a9-03fb653041d1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:25.975921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 402/800] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:25.975921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe935a-b213-4ffe-89a9-03fb653041d1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:25.999748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f307b5-3e9f-44a8-9f0b-9b12a6f02ab0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:26.037226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2ed4e-1ece-4fbc-8916-d461ff9fed61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:26.037865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ed4e-1ece-4fbc-8916-d461ff9fed61 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:26.038147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ed4e-1ece-4fbc-8916-d461ff9fed61 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:26.038422 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ed4e-1ece-4fbc-8916-d461ff9fed61 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:26.039186 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2ed4e-1ece-4fbc-8916-d461ff9fed61 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:26.040496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2ed4e-1ece-4fbc-8916-d461ff9fed61 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:26.048278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-eaf2ed4e-1ece-4fbc-8916-d461ff9fed61 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:26.049765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:26.050776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:26.051612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 399/801] 10.222.0.64 () {56 vars in 1129 bytes} [Thu Aug 4 07:24:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:26.053153 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2ed4e-1ece-4fbc-8916-d461ff9fed61 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:26.092743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:26.099231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe935a-b213-4ffe-89a9-03fb653041d1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:26.137328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68228f3e-9c21-415f-8ad9-ca2a50f7c64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:26.137979 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68228f3e-9c21-415f-8ad9-ca2a50f7c64d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:26.138264 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68228f3e-9c21-415f-8ad9-ca2a50f7c64d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:26.138547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68228f3e-9c21-415f-8ad9-ca2a50f7c64d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:26.139323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68228f3e-9c21-415f-8ad9-ca2a50f7c64d None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=4eedd711c6114de1bfcdf692ce058794) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:26.139631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68228f3e-9c21-415f-8ad9-ca2a50f7c64d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:26.141079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68228f3e-9c21-415f-8ad9-ca2a50f7c64d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:26.155163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 403/802] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:25 2022] GET /identity/v3/role_assignments?user.id=4eedd711c6114de1bfcdf692ce058794&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:26.155163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388ee62-b9a7-472f-b158-ec1983518111 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:26.155163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388ee62-b9a7-472f-b158-ec1983518111 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:26.155163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388ee62-b9a7-472f-b158-ec1983518111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:26.167739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2ed4e-1ece-4fbc-8916-d461ff9fed61 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:26.170350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a388ee62-b9a7-472f-b158-ec1983518111 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:26.172784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a388ee62-b9a7-472f-b158-ec1983518111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:26.210692 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d18df-6f9a-4984-b164-c0f5e804781a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:26.211450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d18df-6f9a-4984-b164-c0f5e804781a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:26.211810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d18df-6f9a-4984-b164-c0f5e804781a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:26.212207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d18df-6f9a-4984-b164-c0f5e804781a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:26.224786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d18df-6f9a-4984-b164-c0f5e804781a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:26.226428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d18df-6f9a-4984-b164-c0f5e804781a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-bf5d18df-6f9a-4984-b164-c0f5e804781a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:26.243123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:26.244209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:26.245126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:26.245126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:26.245126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:26.245126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:26.245126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 400/803] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:26.250010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d18df-6f9a-4984-b164-c0f5e804781a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:26.273326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 404/804] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:26.274276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388ee62-b9a7-472f-b158-ec1983518111 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:26.338770 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:26.378454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:26.391845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d18df-6f9a-4984-b164-c0f5e804781a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:26.400485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388ee62-b9a7-472f-b158-ec1983518111 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:26.441455 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd90cf-fd82-46e6-b100-43a2cf537bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:26.441995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd90cf-fd82-46e6-b100-43a2cf537bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:26.442278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd90cf-fd82-46e6-b100-43a2cf537bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:26.442555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd90cf-fd82-46e6-b100-43a2cf537bd1 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:26.443244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd90cf-fd82-46e6-b100-43a2cf537bd1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:26.443548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd90cf-fd82-46e6-b100-43a2cf537bd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:26.444943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd90cf-fd82-46e6-b100-43a2cf537bd1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:26.449846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de83a78-ca26-4453-add0-5e741256abcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:26.450099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 401/805] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Aug 4 07:24:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:26.450535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de83a78-ca26-4453-add0-5e741256abcf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:26.450823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de83a78-ca26-4453-add0-5e741256abcf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:26.450939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd90cf-fd82-46e6-b100-43a2cf537bd1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:26.451097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de83a78-ca26-4453-add0-5e741256abcf None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:26.451890 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de83a78-ca26-4453-add0-5e741256abcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:26.452263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de83a78-ca26-4453-add0-5e741256abcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:26.453955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de83a78-ca26-4453-add0-5e741256abcf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:26.470009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 405/806] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:26.471719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de83a78-ca26-4453-add0-5e741256abcf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:26.569954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd90cf-fd82-46e6-b100-43a2cf537bd1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:26.580590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de83a78-ca26-4453-add0-5e741256abcf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:26.597807 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:26.609521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17ab0c-d2c7-437e-83dd-3929d0b84985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:26.614234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17ab0c-d2c7-437e-83dd-3929d0b84985 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:26.614234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17ab0c-d2c7-437e-83dd-3929d0b84985 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:26.614234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17ab0c-d2c7-437e-83dd-3929d0b84985 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:26.616847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70a395-1711-4179-bd04-a0d1c6302494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:26.617539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70a395-1711-4179-bd04-a0d1c6302494 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:26.617900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70a395-1711-4179-bd04-a0d1c6302494 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:26.618246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70a395-1711-4179-bd04-a0d1c6302494 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:26.619008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70a395-1711-4179-bd04-a0d1c6302494 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:26.619390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70a395-1711-4179-bd04-a0d1c6302494 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:26.620936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70a395-1711-4179-bd04-a0d1c6302494 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:26.624911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17ab0c-d2c7-437e-83dd-3929d0b84985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:26.630772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 406/807] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Aug 4 07:24:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:26.630772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70a395-1711-4179-bd04-a0d1c6302494 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ed17ab0c-d2c7-437e-83dd-3929d0b84985 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:26.635845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:26.636818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:26.637704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:26.640883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 402/808] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:26.642469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17ab0c-d2c7-437e-83dd-3929d0b84985 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:26.659581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:26.734513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70a395-1711-4179-bd04-a0d1c6302494 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:26.756221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17ab0c-d2c7-437e-83dd-3929d0b84985 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:26.771222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a4179-b285-4883-83aa-8dcb613af5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:26.772045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a4179-b285-4883-83aa-8dcb613af5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:26.772332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a4179-b285-4883-83aa-8dcb613af5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:26.772613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a4179-b285-4883-83aa-8dcb613af5c0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:26.782537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a4179-b285-4883-83aa-8dcb613af5c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:26.784874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a4179-b285-4883-83aa-8dcb613af5c0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:26.797916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76ffa-68fa-4bcb-adff-64cfa63df675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-881a4179-b285-4883-83aa-8dcb613af5c0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:26.798616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:26.800023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:26.801562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:26.801562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:26.801562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:26.801562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:26.801562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:26.801562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:26.801562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:26.801562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:26.801562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:26.801562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:26.802585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76ffa-68fa-4bcb-adff-64cfa63df675 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:26.803082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76ffa-68fa-4bcb-adff-64cfa63df675 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:26.803642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76ffa-68fa-4bcb-adff-64cfa63df675 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:26.808058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 407/809] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Aug 4 07:24:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:26.808058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a4179-b285-4883-83aa-8dcb613af5c0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:26.809269 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c76ffa-68fa-4bcb-adff-64cfa63df675 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:26.811753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c76ffa-68fa-4bcb-adff-64cfa63df675 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:26.853827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:26.920194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 403/810] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:24:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:26.920194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76ffa-68fa-4bcb-adff-64cfa63df675 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:26.927092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a4179-b285-4883-83aa-8dcb613af5c0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:27.015542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a52dc-ee77-404d-b50e-3d5049645a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:27.016176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a52dc-ee77-404d-b50e-3d5049645a80 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.016420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a52dc-ee77-404d-b50e-3d5049645a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.016661 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a52dc-ee77-404d-b50e-3d5049645a80 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.017371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a52dc-ee77-404d-b50e-3d5049645a80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:27.018284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a52dc-ee77-404d-b50e-3d5049645a80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:27.019775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a52dc-ee77-404d-b50e-3d5049645a80 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:27.029706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 408/811] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:27.031097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890076c-edd3-4fc4-8fa9-7f8a47d31a2f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.031400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890076c-edd3-4fc4-8fa9-7f8a47d31a2f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.031683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890076c-edd3-4fc4-8fa9-7f8a47d31a2f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.037682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 409/812] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:27.038453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890076c-edd3-4fc4-8fa9-7f8a47d31a2f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:27.043809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:27.088044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76ffa-68fa-4bcb-adff-64cfa63df675 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:27.136786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aab82a-0057-445a-b4c0-e4772c30ef1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:27.137570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aab82a-0057-445a-b4c0-e4772c30ef1a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.138598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aab82a-0057-445a-b4c0-e4772c30ef1a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.138598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aab82a-0057-445a-b4c0-e4772c30ef1a None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.139208 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aab82a-0057-445a-b4c0-e4772c30ef1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:27.139890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aab82a-0057-445a-b4c0-e4772c30ef1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:27.141234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aab82a-0057-445a-b4c0-e4772c30ef1a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:27.163255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 404/813] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Aug 4 07:24:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:27.163474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:27.168576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890076c-edd3-4fc4-8fa9-7f8a47d31a2f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:27.168707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c238d5-3274-48bb-8c12-33808b258d56 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.174519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c238d5-3274-48bb-8c12-33808b258d56 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.174519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c238d5-3274-48bb-8c12-33808b258d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.188223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-63c238d5-3274-48bb-8c12-33808b258d56 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:27.190111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-63c238d5-3274-48bb-8c12-33808b258d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:27.211918 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a609d11-fe60-4f57-8b92-112fd922f58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:27.212967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a609d11-fe60-4f57-8b92-112fd922f58c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.213456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a609d11-fe60-4f57-8b92-112fd922f58c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.213935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a609d11-fe60-4f57-8b92-112fd922f58c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.225955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a609d11-fe60-4f57-8b92-112fd922f58c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:27.227552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a609d11-fe60-4f57-8b92-112fd922f58c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6a609d11-fe60-4f57-8b92-112fd922f58c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:27.247270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:27.248324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:27.249229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:27.249229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:27.249229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:27.249229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:27.249229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 410/814] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:27.249999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a609d11-fe60-4f57-8b92-112fd922f58c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:27.268332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 405/815] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:27.269331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c238d5-3274-48bb-8c12-33808b258d56 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:27.327221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:27.359553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a609d11-fe60-4f57-8b92-112fd922f58c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:27.373919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c238d5-3274-48bb-8c12-33808b258d56 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:27.396979 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9107ed-d897-407c-860a-72423c34cca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:27.397645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9107ed-d897-407c-860a-72423c34cca3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.397931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9107ed-d897-407c-860a-72423c34cca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.398209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9107ed-d897-407c-860a-72423c34cca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.410917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9107ed-d897-407c-860a-72423c34cca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:27.412554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9107ed-d897-407c-860a-72423c34cca3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:27.413979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6ffaf-ff22-4a4d-a446-71136bfc8907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:27.414613 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6ffaf-ff22-4a4d-a446-71136bfc8907 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.414899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6ffaf-ff22-4a4d-a446-71136bfc8907 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.415175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6ffaf-ff22-4a4d-a446-71136bfc8907 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.415879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6ffaf-ff22-4a4d-a446-71136bfc8907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:27.416222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6ffaf-ff22-4a4d-a446-71136bfc8907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:27.417582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6ffaf-ff22-4a4d-a446-71136bfc8907 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8c9107ed-d897-407c-860a-72423c34cca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:27.429573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:27.431728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:27.433882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:27.433882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:27.433882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:27.433882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:27.434401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 406/817] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:27.434401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6ffaf-ff22-4a4d-a446-71136bfc8907 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:27.436026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 411/817] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:27.437432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9107ed-d897-407c-860a-72423c34cca3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:27.441146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:27.548777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9107ed-d897-407c-860a-72423c34cca3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:27.552237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6ffaf-ff22-4a4d-a446-71136bfc8907 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:27.568744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:27.586150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9082e8-4294-40a2-85f7-22494da04266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:27.587005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9082e8-4294-40a2-85f7-22494da04266 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.587467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9082e8-4294-40a2-85f7-22494da04266 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.587892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9082e8-4294-40a2-85f7-22494da04266 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.588317 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6d069-80c3-4a64-88ec-84f113d2052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:27.589104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6d069-80c3-4a64-88ec-84f113d2052d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.589390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6d069-80c3-4a64-88ec-84f113d2052d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.589666 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6d069-80c3-4a64-88ec-84f113d2052d None admin] PATH_INFO: `/v3/services/network` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.590295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6d069-80c3-4a64-88ec-84f113d2052d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:27.591654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6d069-80c3-4a64-88ec-84f113d2052d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:27.599085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9082e8-4294-40a2-85f7-22494da04266 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:27.600906 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9082e8-4294-40a2-85f7-22494da04266 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-73c6d069-80c3-4a64-88ec-84f113d2052d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:27.602474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:27.603663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:27.604561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 407/818] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:27 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:27.605372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6d069-80c3-4a64-88ec-84f113d2052d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4b9082e8-4294-40a2-85f7-22494da04266 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:27.617297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:27.618259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:27.619260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:27.619260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:27.619260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:27.619260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:27.619260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:27.619260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:27.619260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:27.619260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:27.619260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:27.619260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:27.621725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 412/819] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:27.622646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9082e8-4294-40a2-85f7-22494da04266 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:27.688222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:27.709870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6d069-80c3-4a64-88ec-84f113d2052d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:27.729673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9082e8-4294-40a2-85f7-22494da04266 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:27.746454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5800c-862a-40c3-b3c0-0d5baf014eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:27.747078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5800c-862a-40c3-b3c0-0d5baf014eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.747361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5800c-862a-40c3-b3c0-0d5baf014eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.747635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5800c-862a-40c3-b3c0-0d5baf014eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.748352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5800c-862a-40c3-b3c0-0d5baf014eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:27.748660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5800c-862a-40c3-b3c0-0d5baf014eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:27.750086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5800c-862a-40c3-b3c0-0d5baf014eb1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:27.767700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 408/820] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:27.767700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5800c-862a-40c3-b3c0-0d5baf014eb1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:27.773669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180a48d-7636-4ecc-bdf0-b949e1e936b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:27.774616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a48d-7636-4ecc-bdf0-b949e1e936b7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.775080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a48d-7636-4ecc-bdf0-b949e1e936b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.775507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a48d-7636-4ecc-bdf0-b949e1e936b7 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.776386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180a48d-7636-4ecc-bdf0-b949e1e936b7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:27.776938 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180a48d-7636-4ecc-bdf0-b949e1e936b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:27.778530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180a48d-7636-4ecc-bdf0-b949e1e936b7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:27.784206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 413/821] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:27.785265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180a48d-7636-4ecc-bdf0-b949e1e936b7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:27.809855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:27.871790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5800c-862a-40c3-b3c0-0d5baf014eb1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:27.896256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180a48d-7636-4ecc-bdf0-b949e1e936b7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:27.908523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be4fe4-2049-487f-b66b-bd130546db5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:27.909559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be4fe4-2049-487f-b66b-bd130546db5d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.909987 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be4fe4-2049-487f-b66b-bd130546db5d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.910417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be4fe4-2049-487f-b66b-bd130546db5d None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.911358 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be4fe4-2049-487f-b66b-bd130546db5d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:27.911808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be4fe4-2049-487f-b66b-bd130546db5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:27.913406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be4fe4-2049-487f-b66b-bd130546db5d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:27.923393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:27.932916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a589ffa-9f13-40de-ad64-77f0208e01b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:27.933739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a589ffa-9f13-40de-ad64-77f0208e01b7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:27.933924 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 409/822] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Aug 4 07:24:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:27.940041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a589ffa-9f13-40de-ad64-77f0208e01b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:27.940041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a589ffa-9f13-40de-ad64-77f0208e01b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:27.940254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be4fe4-2049-487f-b66b-bd130546db5d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:27.942738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a589ffa-9f13-40de-ad64-77f0208e01b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:27.944385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a589ffa-9f13-40de-ad64-77f0208e01b7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7a589ffa-9f13-40de-ad64-77f0208e01b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:27.963256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:27.964163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:27.965349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:27.965349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:27.965349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:27.965349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:27.965349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:27.965349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:27.965349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:27.965349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:27.969457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 414/823] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:27.970380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a589ffa-9f13-40de-ad64-77f0208e01b7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:28.047264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:28.052436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be4fe4-2049-487f-b66b-bd130546db5d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:28.078054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a589ffa-9f13-40de-ad64-77f0208e01b7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:28.088847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e82bd-8398-4dd5-8f75-6f0a3f763a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:28.089816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e82bd-8398-4dd5-8f75-6f0a3f763a39 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:28.090297 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e82bd-8398-4dd5-8f75-6f0a3f763a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:28.090769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e82bd-8398-4dd5-8f75-6f0a3f763a39 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:28.091647 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e82bd-8398-4dd5-8f75-6f0a3f763a39 None admin] RBAC: Adding query filter params (type=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:28.092203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e82bd-8398-4dd5-8f75-6f0a3f763a39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:28.093758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e82bd-8398-4dd5-8f75-6f0a3f763a39 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:28.099947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 410/824] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:28.101083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e82bd-8398-4dd5-8f75-6f0a3f763a39 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:28.117523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96112102-6d69-4d60-8107-4c6712a0129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:28.118171 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96112102-6d69-4d60-8107-4c6712a0129e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:28.118498 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96112102-6d69-4d60-8107-4c6712a0129e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:28.118780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96112102-6d69-4d60-8107-4c6712a0129e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:28.130228 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96112102-6d69-4d60-8107-4c6712a0129e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:28.131716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96112102-6d69-4d60-8107-4c6712a0129e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-96112102-6d69-4d60-8107-4c6712a0129e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:28.147039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:28.147950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:28.149458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:28.149458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:28.149458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:28.149458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:28.149458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 415/825] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:28.149458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96112102-6d69-4d60-8107-4c6712a0129e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:28.192480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:28.211472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e82bd-8398-4dd5-8f75-6f0a3f763a39 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:28.247683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210417b-d1a7-4ddd-a743-7ef411b9003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:28.248471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210417b-d1a7-4ddd-a743-7ef411b9003e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:28.248812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210417b-d1a7-4ddd-a743-7ef411b9003e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:28.249147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210417b-d1a7-4ddd-a743-7ef411b9003e None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:28.249979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210417b-d1a7-4ddd-a743-7ef411b9003e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:28.250530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210417b-d1a7-4ddd-a743-7ef411b9003e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:28.252016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210417b-d1a7-4ddd-a743-7ef411b9003e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:28.261422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96112102-6d69-4d60-8107-4c6712a0129e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:28.270801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 411/826] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:28.270801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210417b-d1a7-4ddd-a743-7ef411b9003e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:28.293962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:28.307729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9f2e3-7be1-453c-9e84-667d0de2dea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:28.308400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9f2e3-7be1-453c-9e84-667d0de2dea1 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:28.308685 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9f2e3-7be1-453c-9e84-667d0de2dea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:28.309061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9f2e3-7be1-453c-9e84-667d0de2dea1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:28.309844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9f2e3-7be1-453c-9e84-667d0de2dea1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:28.311306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9f2e3-7be1-453c-9e84-667d0de2dea1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:28.373762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210417b-d1a7-4ddd-a743-7ef411b9003e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:28.406477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db260f-552f-4e38-87c7-8f9306a27f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:28.407084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db260f-552f-4e38-87c7-8f9306a27f4a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:28.407322 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 416/827] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:24:28 2022] POST /identity/v3/endpoints => generated 337 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:28.407428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db260f-552f-4e38-87c7-8f9306a27f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:28.407624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db260f-552f-4e38-87c7-8f9306a27f4a None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:28.409169 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9f2e3-7be1-453c-9e84-667d0de2dea1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:28.409296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db260f-552f-4e38-87c7-8f9306a27f4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:28.409522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db260f-552f-4e38-87c7-8f9306a27f4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:28.410912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db260f-552f-4e38-87c7-8f9306a27f4a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:28.428559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 412/828] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:28.429919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f339be-69d4-4e05-9918-a68108f2c8e9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:28.430204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f339be-69d4-4e05-9918-a68108f2c8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:28.430477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f339be-69d4-4e05-9918-a68108f2c8e9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:28.431937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 413/829] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:28.440436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f339be-69d4-4e05-9918-a68108f2c8e9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:28.530864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:28.547306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9f2e3-7be1-453c-9e84-667d0de2dea1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:28.556748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:28.581877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f339be-69d4-4e05-9918-a68108f2c8e9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:28.594178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287c791-79dd-4810-8796-cadf442eb259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:28.594861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c791-79dd-4810-8796-cadf442eb259 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:28.595142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c791-79dd-4810-8796-cadf442eb259 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:28.595467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c791-79dd-4810-8796-cadf442eb259 None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/0c30c2fd1c4648308fac6606526451ec/roles/88637a5a7c9d47aaacf803a09abc1755` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:28.614911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287c791-79dd-4810-8796-cadf442eb259 None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=0c30c2fd1c4648308fac6606526451ec, role_id=88637a5a7c9d47aaacf803a09abc1755)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:28.616401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287c791-79dd-4810-8796-cadf442eb259 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:28.628721 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644599e7-1db1-4c35-910b-9ecab77fa47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:28.629300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644599e7-1db1-4c35-910b-9ecab77fa47a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:28.629582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644599e7-1db1-4c35-910b-9ecab77fa47a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:28.629859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644599e7-1db1-4c35-910b-9ecab77fa47a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:28.637945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644599e7-1db1-4c35-910b-9ecab77fa47a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:28.639465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644599e7-1db1-4c35-910b-9ecab77fa47a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-644599e7-1db1-4c35-910b-9ecab77fa47a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:28.652377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:28.653275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:28.654200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:28.654200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:28.654200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:28.654200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:28.654200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:28.654200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:28.654200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:28.654200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:28.654200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 414/830] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:28.655766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe0c85-0415-4c7a-9a75-3775e17a4927 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:28.655766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe0c85-0415-4c7a-9a75-3775e17a4927 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:28.655766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe0c85-0415-4c7a-9a75-3775e17a4927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:28.674169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fafe0c85-0415-4c7a-9a75-3775e17a4927 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:28.675368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fafe0c85-0415-4c7a-9a75-3775e17a4927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:28.687823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 417/831] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:24:28 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/0c30c2fd1c4648308fac6606526451ec/roles/88637a5a7c9d47aaacf803a09abc1755 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:24:28.688665 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287c791-79dd-4810-8796-cadf442eb259 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:28.764102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 415/832] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:28.786993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe0c85-0415-4c7a-9a75-3775e17a4927 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:28.817237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287c791-79dd-4810-8796-cadf442eb259 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:28.856683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dce8f9-068d-42dc-a189-8ff460a336c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:28.857614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dce8f9-068d-42dc-a189-8ff460a336c0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:28.857901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dce8f9-068d-42dc-a189-8ff460a336c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:28.858176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dce8f9-068d-42dc-a189-8ff460a336c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:28.858942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dce8f9-068d-42dc-a189-8ff460a336c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:28.859250 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dce8f9-068d-42dc-a189-8ff460a336c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:28.860773 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dce8f9-068d-42dc-a189-8ff460a336c0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:28.872248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:28.878546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 418/833] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:28.887790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dce8f9-068d-42dc-a189-8ff460a336c0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:28.889548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:28.927688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe0c85-0415-4c7a-9a75-3775e17a4927 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:28.975665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28794ce-195c-47ef-bbd5-e9a266bb4c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:28.976347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28794ce-195c-47ef-bbd5-e9a266bb4c50 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:28.976626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28794ce-195c-47ef-bbd5-e9a266bb4c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:28.977020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28794ce-195c-47ef-bbd5-e9a266bb4c50 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:28.977685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28794ce-195c-47ef-bbd5-e9a266bb4c50 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:28.979137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28794ce-195c-47ef-bbd5-e9a266bb4c50 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c28794ce-195c-47ef-bbd5-e9a266bb4c50 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:28.989367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:28.990538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:28.991591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 416/834] 10.222.0.64 () {56 vars in 1132 bytes} [Thu Aug 4 07:24:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:28.999848 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28794ce-195c-47ef-bbd5-e9a266bb4c50 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:29.014036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dce8f9-068d-42dc-a189-8ff460a336c0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:29.049679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10db68-d800-4b9e-b5b3-a093b578a95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:29.050397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10db68-d800-4b9e-b5b3-a093b578a95e None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:29.050728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10db68-d800-4b9e-b5b3-a093b578a95e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:29.051010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10db68-d800-4b9e-b5b3-a093b578a95e None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/9503e2888d5e4831afc0d9d9522c3eea/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:29.053020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10db68-d800-4b9e-b5b3-a093b578a95e None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=9503e2888d5e4831afc0d9d9522c3eea, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:29.054541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10db68-d800-4b9e-b5b3-a093b578a95e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:29.109136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28794ce-195c-47ef-bbd5-e9a266bb4c50 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:29.122145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 419/835] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:24:28 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/9503e2888d5e4831afc0d9d9522c3eea/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:24:29.161809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833ebcb-9b54-423e-ad46-9451a477edae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:29.161809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833ebcb-9b54-423e-ad46-9451a477edae None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:29.161809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833ebcb-9b54-423e-ad46-9451a477edae None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:29.161809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833ebcb-9b54-423e-ad46-9451a477edae None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:29.162214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833ebcb-9b54-423e-ad46-9451a477edae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:29.162214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833ebcb-9b54-423e-ad46-9451a477edae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:29.162214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833ebcb-9b54-423e-ad46-9451a477edae None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:29.164544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:29.169406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 417/836] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Aug 4 07:24:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:29.174930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10db68-d800-4b9e-b5b3-a093b578a95e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:29.212565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf4f81-2854-4173-a9e8-69a85767a561 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:29.213444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf4f81-2854-4173-a9e8-69a85767a561 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:29.214142 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf4f81-2854-4173-a9e8-69a85767a561 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:29.220018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 418/837] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:24:29.223941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d06d0d-ca0c-4511-900c-8a4a3bcb091e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:29.224268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d06d0d-ca0c-4511-900c-8a4a3bcb091e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:29.224546 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d06d0d-ca0c-4511-900c-8a4a3bcb091e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:29.245623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-15d06d0d-ca0c-4511-900c-8a4a3bcb091e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:29.246853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-15d06d0d-ca0c-4511-900c-8a4a3bcb091e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:29.258635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:29.319263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10db68-d800-4b9e-b5b3-a093b578a95e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:29.334730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 419/838] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:29.357416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfc939-9229-4db2-850e-741e901e214e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:29.358043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfc939-9229-4db2-850e-741e901e214e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:29.358333 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfc939-9229-4db2-850e-741e901e214e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:29.358615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfc939-9229-4db2-850e-741e901e214e None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:29.359307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfc939-9229-4db2-850e-741e901e214e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:29.359616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfc939-9229-4db2-850e-741e901e214e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:29.362090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfc939-9229-4db2-850e-741e901e214e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:29.362862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d06d0d-ca0c-4511-900c-8a4a3bcb091e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:29.367444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 420/839] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Aug 4 07:24:29 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:29.480159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d06d0d-ca0c-4511-900c-8a4a3bcb091e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:29.513879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:29.520792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99649b45-d868-4795-b51b-fa24d36b1657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:29.521499 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99649b45-d868-4795-b51b-fa24d36b1657 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:29.521788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99649b45-d868-4795-b51b-fa24d36b1657 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:29.522157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99649b45-d868-4795-b51b-fa24d36b1657 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:29.522810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99649b45-d868-4795-b51b-fa24d36b1657 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:29.524292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99649b45-d868-4795-b51b-fa24d36b1657 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-99649b45-d868-4795-b51b-fa24d36b1657 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:29.535023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:29.536091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:29.536931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 420/840] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:24:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:29.542592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfc939-9229-4db2-850e-741e901e214e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:29.649384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfc939-9229-4db2-850e-741e901e214e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:29.672959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:29.684320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c0500-730f-4c66-87de-45bddf54f2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:29.684942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c0500-730f-4c66-87de-45bddf54f2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:29.685227 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c0500-730f-4c66-87de-45bddf54f2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:29.685502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c0500-730f-4c66-87de-45bddf54f2f6 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:29.686186 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c0500-730f-4c66-87de-45bddf54f2f6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:29.686492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c0500-730f-4c66-87de-45bddf54f2f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:29.687899 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c0500-730f-4c66-87de-45bddf54f2f6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:29.692930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 421/841] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:24:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:29.698491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99649b45-d868-4795-b51b-fa24d36b1657 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:29.804826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99649b45-d868-4795-b51b-fa24d36b1657 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:29.840995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469122bf-545c-4454-accb-2989ad927662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:29.842089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469122bf-545c-4454-accb-2989ad927662 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:29.842577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469122bf-545c-4454-accb-2989ad927662 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:29.843058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469122bf-545c-4454-accb-2989ad927662 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:29.843956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469122bf-545c-4454-accb-2989ad927662 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:29.844496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469122bf-545c-4454-accb-2989ad927662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:29.847035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469122bf-545c-4454-accb-2989ad927662 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:29.852523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 421/842] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:24:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:30.819555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9a641-b18d-4e29-911b-1cafe8608ff2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:30.820088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9a641-b18d-4e29-911b-1cafe8608ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:30.820165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9a641-b18d-4e29-911b-1cafe8608ff2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:30.821748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 422/843] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:30.827100 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c24e3e-c0e8-4ecf-9800-2727d0cb7e66 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:30.827503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c24e3e-c0e8-4ecf-9800-2727d0cb7e66 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:30.827684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c24e3e-c0e8-4ecf-9800-2727d0cb7e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:30.846729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f6c24e3e-c0e8-4ecf-9800-2727d0cb7e66 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:30.847916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f6c24e3e-c0e8-4ecf-9800-2727d0cb7e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:30.903216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:30.920212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 422/844] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:30.940706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9a641-b18d-4e29-911b-1cafe8608ff2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:31.054564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9a641-b18d-4e29-911b-1cafe8608ff2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:31.090437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2833ed-6592-49a2-9cfe-c1701a19c946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:31.090999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2833ed-6592-49a2-9cfe-c1701a19c946 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.091281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2833ed-6592-49a2-9cfe-c1701a19c946 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.091555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2833ed-6592-49a2-9cfe-c1701a19c946 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.102134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2833ed-6592-49a2-9cfe-c1701a19c946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fc2833ed-6592-49a2-9cfe-c1701a19c946 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:31.107279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:31.108229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:31.109079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 423/845] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:24:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:31.114948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c24e3e-c0e8-4ecf-9800-2727d0cb7e66 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:31.219178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c24e3e-c0e8-4ecf-9800-2727d0cb7e66 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:31.253584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395360b-7e0b-49d9-96e0-9866546b2396 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.253923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395360b-7e0b-49d9-96e0-9866546b2396 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.254141 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395360b-7e0b-49d9-96e0-9866546b2396 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.256156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2219e47-c960-4fa7-bd4c-fbf388836f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:31.256783 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2219e47-c960-4fa7-bd4c-fbf388836f38 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.257065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2219e47-c960-4fa7-bd4c-fbf388836f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.257342 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2219e47-c960-4fa7-bd4c-fbf388836f38 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.258065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2219e47-c960-4fa7-bd4c-fbf388836f38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:31.258367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2219e47-c960-4fa7-bd4c-fbf388836f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:31.259439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 424/846] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:31.259903 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2219e47-c960-4fa7-bd4c-fbf388836f38 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:31.260909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37497ebb-9742-45cb-8d0e-2881b9eed557 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.261194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37497ebb-9742-45cb-8d0e-2881b9eed557 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.261472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37497ebb-9742-45cb-8d0e-2881b9eed557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.269214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 423/847] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:24:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:31.276971 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2219e47-c960-4fa7-bd4c-fbf388836f38 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:31.284523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-37497ebb-9742-45cb-8d0e-2881b9eed557 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:31.285826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-37497ebb-9742-45cb-8d0e-2881b9eed557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:31.331235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:31.352402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:31.387443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 425/848] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:31.392702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2219e47-c960-4fa7-bd4c-fbf388836f38 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:31.410899 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37497ebb-9742-45cb-8d0e-2881b9eed557 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:31.431535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedc469-9f19-400d-92d2-7fa0c6ae5f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:31.432543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedc469-9f19-400d-92d2-7fa0c6ae5f0a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.433048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedc469-9f19-400d-92d2-7fa0c6ae5f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.433538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedc469-9f19-400d-92d2-7fa0c6ae5f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.449618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedc469-9f19-400d-92d2-7fa0c6ae5f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:31.451643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedc469-9f19-400d-92d2-7fa0c6ae5f0a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7bedc469-9f19-400d-92d2-7fa0c6ae5f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:31.471434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:31.472428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:31.473327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:31.473327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:31.473327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:31.473327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:31.481042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 424/849] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:31.482844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedc469-9f19-400d-92d2-7fa0c6ae5f0a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:31.532035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37497ebb-9742-45cb-8d0e-2881b9eed557 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:31.574398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb09f8-5c13-48ef-9e16-97c6584b294e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:31.575057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb09f8-5c13-48ef-9e16-97c6584b294e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.575341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb09f8-5c13-48ef-9e16-97c6584b294e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.575616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb09f8-5c13-48ef-9e16-97c6584b294e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.586281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb09f8-5c13-48ef-9e16-97c6584b294e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d5cb09f8-5c13-48ef-9e16-97c6584b294e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:31.591172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:31.592134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:31.592989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 426/850] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:31.593678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe1cc1-8833-4575-ac47-2267caf93c90 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.594241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe1cc1-8833-4575-ac47-2267caf93c90 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.594241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe1cc1-8833-4575-ac47-2267caf93c90 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.595465 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 427/851] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:31.599921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe1cc1-8833-4575-ac47-2267caf93c90 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:31.605335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedc469-9f19-400d-92d2-7fa0c6ae5f0a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:31.607264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:31.646857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a2bff-d9fc-4b4a-beec-6dbdda0346cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:31.647879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2bff-d9fc-4b4a-beec-6dbdda0346cb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.648474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:31.648778 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2bff-d9fc-4b4a-beec-6dbdda0346cb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.649072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2bff-d9fc-4b4a-beec-6dbdda0346cb None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.649852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a2bff-d9fc-4b4a-beec-6dbdda0346cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:31.650094 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a2bff-d9fc-4b4a-beec-6dbdda0346cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:31.651868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a2bff-d9fc-4b4a-beec-6dbdda0346cb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:31.671635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 425/852] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:31.671635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c952543-602c-423b-aa79-ac09767a3e6f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.671635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c952543-602c-423b-aa79-ac09767a3e6f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.671635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c952543-602c-423b-aa79-ac09767a3e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.690391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4c952543-602c-423b-aa79-ac09767a3e6f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:31.691571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4c952543-602c-423b-aa79-ac09767a3e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:31.713502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe1cc1-8833-4575-ac47-2267caf93c90 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:31.749240 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9848298-dcfc-45cc-a8d1-c3ca5df08c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:31.749859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9848298-dcfc-45cc-a8d1-c3ca5df08c49 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.750142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9848298-dcfc-45cc-a8d1-c3ca5df08c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.750421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9848298-dcfc-45cc-a8d1-c3ca5df08c49 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.751137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9848298-dcfc-45cc-a8d1-c3ca5df08c49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:31.751446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9848298-dcfc-45cc-a8d1-c3ca5df08c49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:31.752882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9848298-dcfc-45cc-a8d1-c3ca5df08c49 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:31.761037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 428/853] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:31.761815 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9848298-dcfc-45cc-a8d1-c3ca5df08c49 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:31.770219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 426/854] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:31.771336 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c952543-602c-423b-aa79-ac09767a3e6f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:31.873931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:31.875797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:31.879544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9848298-dcfc-45cc-a8d1-c3ca5df08c49 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:31.888606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c952543-602c-423b-aa79-ac09767a3e6f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:31.917197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7620899-33d5-4e48-a1f5-bfe2e0a86526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:31.917938 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7620899-33d5-4e48-a1f5-bfe2e0a86526 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.918215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7620899-33d5-4e48-a1f5-bfe2e0a86526 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.918608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7620899-33d5-4e48-a1f5-bfe2e0a86526 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.926297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7620899-33d5-4e48-a1f5-bfe2e0a86526 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:31.927196 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c55a14-c558-45ad-9a11-7b56c2ffc63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:31.927730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7620899-33d5-4e48-a1f5-bfe2e0a86526 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:31.927847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c55a14-c558-45ad-9a11-7b56c2ffc63b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:31.928183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c55a14-c558-45ad-9a11-7b56c2ffc63b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:31.928418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c55a14-c558-45ad-9a11-7b56c2ffc63b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:31.938471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c55a14-c558-45ad-9a11-7b56c2ffc63b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:31.940472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c55a14-c558-45ad-9a11-7b56c2ffc63b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f7620899-33d5-4e48-a1f5-bfe2e0a86526 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:31.940876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:31.941772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 429/855] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Aug 4 07:24:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:31.942700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7620899-33d5-4e48-a1f5-bfe2e0a86526 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-80c55a14-c558-45ad-9a11-7b56c2ffc63b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:31.963735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:31.964706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:31.965592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:31.965592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:31.965592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:31.965592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:31.965592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 427/856] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:31.965866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c55a14-c558-45ad-9a11-7b56c2ffc63b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:32.048113 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7620899-33d5-4e48-a1f5-bfe2e0a86526 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:32.068382 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c55a14-c558-45ad-9a11-7b56c2ffc63b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:32.082065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acef345-5155-456a-9c05-9b402a4888d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:32.082673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acef345-5155-456a-9c05-9b402a4888d3 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:32.082946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acef345-5155-456a-9c05-9b402a4888d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:32.083232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acef345-5155-456a-9c05-9b402a4888d3 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:32.083895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acef345-5155-456a-9c05-9b402a4888d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:32.085274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acef345-5155-456a-9c05-9b402a4888d3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:32.100987 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c8f4c-d939-4434-a3aa-cc5880a7004a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:32.101599 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c8f4c-d939-4434-a3aa-cc5880a7004a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:32.101882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c8f4c-d939-4434-a3aa-cc5880a7004a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:32.102156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c8f4c-d939-4434-a3aa-cc5880a7004a None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:32.102949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c8f4c-d939-4434-a3aa-cc5880a7004a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:32.103257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c8f4c-d939-4434-a3aa-cc5880a7004a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:32.104692 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c8f4c-d939-4434-a3aa-cc5880a7004a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:32.122182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 428/857] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Aug 4 07:24:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:32.123339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99970a81-e46d-450d-9d9c-1f658d55a6a8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:32.123626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99970a81-e46d-450d-9d9c-1f658d55a6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:32.123904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99970a81-e46d-450d-9d9c-1f658d55a6a8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:32.125175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 429/858] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:32.125838 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99970a81-e46d-450d-9d9c-1f658d55a6a8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:32.129410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:32.156367 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 430/859] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:24:31 2022] POST /identity/v3/services => generated 269 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:32.157175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acef345-5155-456a-9c05-9b402a4888d3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:32.185352 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:32.258792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99970a81-e46d-450d-9d9c-1f658d55a6a8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:32.275502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acef345-5155-456a-9c05-9b402a4888d3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:32.297655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754c73b-1dc0-44c0-a398-dc365e270f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:32.298284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754c73b-1dc0-44c0-a398-dc365e270f25 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:32.299150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754c73b-1dc0-44c0-a398-dc365e270f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:32.299432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754c73b-1dc0-44c0-a398-dc365e270f25 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:32.300201 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754c73b-1dc0-44c0-a398-dc365e270f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:32.300514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754c73b-1dc0-44c0-a398-dc365e270f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:32.301894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754c73b-1dc0-44c0-a398-dc365e270f25 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:32.317662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 430/860] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:32.318761 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d63c1-3769-4e7e-af16-55e2b91c859b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:32.319105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d63c1-3769-4e7e-af16-55e2b91c859b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:32.319388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d63c1-3769-4e7e-af16-55e2b91c859b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:32.322706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cb481-3389-452b-a862-f7bcc1c1c126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:32.323734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cb481-3389-452b-a862-f7bcc1c1c126 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:32.323734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cb481-3389-452b-a862-f7bcc1c1c126 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:32.323899 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cb481-3389-452b-a862-f7bcc1c1c126 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:32.337931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-811d63c1-3769-4e7e-af16-55e2b91c859b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:32.339789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-811d63c1-3769-4e7e-af16-55e2b91c859b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:32.396742 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cb481-3389-452b-a862-f7bcc1c1c126 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:32.398113 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cb481-3389-452b-a862-f7bcc1c1c126 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:32.405671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:32.420732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 431/861] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:32 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:32.420967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:32.421876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d63c1-3769-4e7e-af16-55e2b91c859b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:32.513284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 431/862] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:32 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:24:32.514103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cb481-3389-452b-a862-f7bcc1c1c126 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:32.540593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d63c1-3769-4e7e-af16-55e2b91c859b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:32.576754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76872ef2-b15a-449b-886c-f1396f7e2745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:32.577403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76872ef2-b15a-449b-886c-f1396f7e2745 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:32.577687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76872ef2-b15a-449b-886c-f1396f7e2745 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:32.577968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76872ef2-b15a-449b-886c-f1396f7e2745 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:32.585334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76872ef2-b15a-449b-886c-f1396f7e2745 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:32.586817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76872ef2-b15a-449b-886c-f1396f7e2745 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-76872ef2-b15a-449b-886c-f1396f7e2745 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:32.600141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:32.601129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 432/863] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:32.602095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76872ef2-b15a-449b-886c-f1396f7e2745 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:32.626311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cb481-3389-452b-a862-f7bcc1c1c126 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:32.649750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:32.656596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:32.666033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ee1ae-d7b2-489a-b93f-ccef533c4a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:32.666646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ee1ae-d7b2-489a-b93f-ccef533c4a0b None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:32.666927 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ee1ae-d7b2-489a-b93f-ccef533c4a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:32.667197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ee1ae-d7b2-489a-b93f-ccef533c4a0b None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:32.667887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ee1ae-d7b2-489a-b93f-ccef533c4a0b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:32.669295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ee1ae-d7b2-489a-b93f-ccef533c4a0b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:32.708633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76872ef2-b15a-449b-886c-f1396f7e2745 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:32.742645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eea466-7b00-49a5-8782-5ab9c49b0e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:32.743286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eea466-7b00-49a5-8782-5ab9c49b0e71 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:32.743663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eea466-7b00-49a5-8782-5ab9c49b0e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:32.743949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eea466-7b00-49a5-8782-5ab9c49b0e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:32.754671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eea466-7b00-49a5-8782-5ab9c49b0e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:32.756211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eea466-7b00-49a5-8782-5ab9c49b0e71 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:32.768290 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 432/864] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:24:32 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:32.768290 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ee1ae-d7b2-489a-b93f-ccef533c4a0b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-42eea466-7b00-49a5-8782-5ab9c49b0e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:32.786071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:32.787278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:32.788307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:32.788307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:32.788307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:32.788307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:32.788307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 433/865] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:32.793150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eea466-7b00-49a5-8782-5ab9c49b0e71 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:32.906456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ee1ae-d7b2-489a-b93f-ccef533c4a0b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:32.921889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:32.926933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eea466-7b00-49a5-8782-5ab9c49b0e71 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:32.951756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8dc614-23a0-42b5-b436-474ceb4715d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:32.952792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dc614-23a0-42b5-b436-474ceb4715d5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:32.953312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dc614-23a0-42b5-b436-474ceb4715d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:32.953593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dc614-23a0-42b5-b436-474ceb4715d5 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:32.955802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8dc614-23a0-42b5-b436-474ceb4715d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:32.955802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8dc614-23a0-42b5-b436-474ceb4715d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:32.959472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8dc614-23a0-42b5-b436-474ceb4715d5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:32.972583 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755baf25-6fe0-4e70-9606-8194dccff9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:32.973207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755baf25-6fe0-4e70-9606-8194dccff9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:32.973488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755baf25-6fe0-4e70-9606-8194dccff9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:32.973762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755baf25-6fe0-4e70-9606-8194dccff9c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:32.974460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755baf25-6fe0-4e70-9606-8194dccff9c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:32.974762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755baf25-6fe0-4e70-9606-8194dccff9c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:32.976886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755baf25-6fe0-4e70-9606-8194dccff9c5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:32.983964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 433/866] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Aug 4 07:24:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:32.992205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8dc614-23a0-42b5-b436-474ceb4715d5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:32.992360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 434/867] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:32.997376 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755baf25-6fe0-4e70-9606-8194dccff9c5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:33.028935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:33.107971 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755baf25-6fe0-4e70-9606-8194dccff9c5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:33.114701 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8dc614-23a0-42b5-b436-474ceb4715d5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:33.146376 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555524a-a09e-4cb3-8853-2884c4f79e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:33.147017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555524a-a09e-4cb3-8853-2884c4f79e88 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:33.147295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555524a-a09e-4cb3-8853-2884c4f79e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:33.147566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555524a-a09e-4cb3-8853-2884c4f79e88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:33.154984 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555524a-a09e-4cb3-8853-2884c4f79e88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:33.156720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555524a-a09e-4cb3-8853-2884c4f79e88 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:33.161762 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38e03b-272a-4b77-89c3-e5f59d900e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:33.162668 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38e03b-272a-4b77-89c3-e5f59d900e38 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:33.163182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38e03b-272a-4b77-89c3-e5f59d900e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:33.163675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38e03b-272a-4b77-89c3-e5f59d900e38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b555524a-a09e-4cb3-8853-2884c4f79e88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:33.168706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:33.169609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:33.170503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:33.170503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:33.170503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:33.170503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:33.170503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:33.170503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:33.170503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:33.170503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:33.174633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 435/868] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:33.175890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555524a-a09e-4cb3-8853-2884c4f79e88 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:33.200174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:33.240176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38e03b-272a-4b77-89c3-e5f59d900e38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:33.242058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38e03b-272a-4b77-89c3-e5f59d900e38 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:33.280204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:33.288435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555524a-a09e-4cb3-8853-2884c4f79e88 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:33.339027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574538a1-c925-48c5-bb1c-c0ba14985e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:33.340480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574538a1-c925-48c5-bb1c-c0ba14985e3d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:33.341005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574538a1-c925-48c5-bb1c-c0ba14985e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:33.341491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574538a1-c925-48c5-bb1c-c0ba14985e3d None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:33.342994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574538a1-c925-48c5-bb1c-c0ba14985e3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:33.343519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574538a1-c925-48c5-bb1c-c0ba14985e3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:33.345738 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574538a1-c925-48c5-bb1c-c0ba14985e3d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:33.361078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 434/869] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:32 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:24:33.362171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 436/870] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:33.362577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38e03b-272a-4b77-89c3-e5f59d900e38 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:33.373595 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574538a1-c925-48c5-bb1c-c0ba14985e3d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:33.468763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38e03b-272a-4b77-89c3-e5f59d900e38 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:33.476360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:33.504692 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574538a1-c925-48c5-bb1c-c0ba14985e3d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:33.506750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5124806-2b5c-4c0d-891b-020493d62b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:33.507453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5124806-2b5c-4c0d-891b-020493d62b0e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:33.507803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5124806-2b5c-4c0d-891b-020493d62b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:33.508188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5124806-2b5c-4c0d-891b-020493d62b0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:33.510301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:33.522083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5124806-2b5c-4c0d-891b-020493d62b0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:33.523559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5124806-2b5c-4c0d-891b-020493d62b0e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c5124806-2b5c-4c0d-891b-020493d62b0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:33.541179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:33.542096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:33.542977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:33.542977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:33.542977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:33.542977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:33.547015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 435/871] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:33.553546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5124806-2b5c-4c0d-891b-020493d62b0e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:33.558812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ef788-4e16-48a5-af80-e5bc40fc6a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:33.559659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ef788-4e16-48a5-af80-e5bc40fc6a26 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:33.560191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ef788-4e16-48a5-af80-e5bc40fc6a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:33.561041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ef788-4e16-48a5-af80-e5bc40fc6a26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:33.562364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ef788-4e16-48a5-af80-e5bc40fc6a26 None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=0c30c2fd1c4648308fac6606526451ec) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:33.563164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ef788-4e16-48a5-af80-e5bc40fc6a26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:33.566380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ef788-4e16-48a5-af80-e5bc40fc6a26 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:33.580261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 437/872] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:33 2022] GET /identity/v3/role_assignments?user.id=0c30c2fd1c4648308fac6606526451ec&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:33.665665 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5124806-2b5c-4c0d-891b-020493d62b0e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:33.704347 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedf807-d031-4256-8491-fd4896c82ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:33.705103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedf807-d031-4256-8491-fd4896c82ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:33.705386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedf807-d031-4256-8491-fd4896c82ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:33.705662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedf807-d031-4256-8491-fd4896c82ed2 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:33.706349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedf807-d031-4256-8491-fd4896c82ed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:33.706651 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedf807-d031-4256-8491-fd4896c82ed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:33.708048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedf807-d031-4256-8491-fd4896c82ed2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:33.726235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 436/873] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:33.727664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ef788-4e16-48a5-af80-e5bc40fc6a26 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:33.835442 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ef788-4e16-48a5-af80-e5bc40fc6a26 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:33.872211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b2541-eef2-4e62-acc3-1efa82882263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:33.872856 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b2541-eef2-4e62-acc3-1efa82882263 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:33.873134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b2541-eef2-4e62-acc3-1efa82882263 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:33.873405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b2541-eef2-4e62-acc3-1efa82882263 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:33.880937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b2541-eef2-4e62-acc3-1efa82882263 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:33.882364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b2541-eef2-4e62-acc3-1efa82882263 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-247b2541-eef2-4e62-acc3-1efa82882263 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:33.894233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:33.895168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:33.896105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:33.896105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:33.896105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:33.896105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:33.896105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:33.896105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:33.896105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:33.896105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:33.900068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 438/874] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:33.901603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedf807-d031-4256-8491-fd4896c82ed2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:33.954402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:34.011083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedf807-d031-4256-8491-fd4896c82ed2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:34.047027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e976bb8-0f94-4a44-a83f-21561dd63118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:34.047641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e976bb8-0f94-4a44-a83f-21561dd63118 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:34.047918 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e976bb8-0f94-4a44-a83f-21561dd63118 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:34.048217 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e976bb8-0f94-4a44-a83f-21561dd63118 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:34.049026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e976bb8-0f94-4a44-a83f-21561dd63118 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:34.049330 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e976bb8-0f94-4a44-a83f-21561dd63118 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:34.050756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e976bb8-0f94-4a44-a83f-21561dd63118 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:34.066932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 437/875] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:34.069854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b2541-eef2-4e62-acc3-1efa82882263 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:34.073728 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:34.178176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b2541-eef2-4e62-acc3-1efa82882263 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:34.213873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca85fa-49c0-4085-a1e8-0123b1a4a5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:34.214555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca85fa-49c0-4085-a1e8-0123b1a4a5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:34.214844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca85fa-49c0-4085-a1e8-0123b1a4a5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:34.215123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca85fa-49c0-4085-a1e8-0123b1a4a5c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:34.215920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca85fa-49c0-4085-a1e8-0123b1a4a5c6 None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=9503e2888d5e4831afc0d9d9522c3eea) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:34.216335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca85fa-49c0-4085-a1e8-0123b1a4a5c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:34.217759 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca85fa-49c0-4085-a1e8-0123b1a4a5c6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:34.224220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 439/876] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:34 2022] GET /identity/v3/role_assignments?user.id=9503e2888d5e4831afc0d9d9522c3eea&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:34.233264 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078eae0-227f-433f-bcbd-c204fee369a2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:34.233765 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078eae0-227f-433f-bcbd-c204fee369a2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:34.234047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078eae0-227f-433f-bcbd-c204fee369a2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:34.235803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 438/877] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:34.242452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0ba89-8c56-40e3-9494-9a4896d9cd41 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:34.242764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0ba89-8c56-40e3-9494-9a4896d9cd41 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:34.243008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0ba89-8c56-40e3-9494-9a4896d9cd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:34.263113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2fe0ba89-8c56-40e3-9494-9a4896d9cd41 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:34.264349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2fe0ba89-8c56-40e3-9494-9a4896d9cd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:34.333595 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:34.339847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 440/878] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:34.365221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078eae0-227f-433f-bcbd-c204fee369a2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:34.474987 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078eae0-227f-433f-bcbd-c204fee369a2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:34.483019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:34.511004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ab905-0980-4204-b4a3-95a30b737fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:34.511648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ab905-0980-4204-b4a3-95a30b737fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:34.511998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ab905-0980-4204-b4a3-95a30b737fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:34.512290 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ab905-0980-4204-b4a3-95a30b737fc7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:34.513110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ab905-0980-4204-b4a3-95a30b737fc7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:34.514540 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ab905-0980-4204-b4a3-95a30b737fc7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9c6ab905-0980-4204-b4a3-95a30b737fc7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:34.524955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:34.527197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:34.529312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 439/879] 10.222.0.64 () {56 vars in 1132 bytes} [Thu Aug 4 07:24:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:34.534008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0ba89-8c56-40e3-9494-9a4896d9cd41 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:34.637776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0ba89-8c56-40e3-9494-9a4896d9cd41 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:34.674587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fc8ca-54ab-4aee-ab95-8fad308718a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:34.675268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fc8ca-54ab-4aee-ab95-8fad308718a4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:34.675545 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fc8ca-54ab-4aee-ab95-8fad308718a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:34.675815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fc8ca-54ab-4aee-ab95-8fad308718a4 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:34.676524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fc8ca-54ab-4aee-ab95-8fad308718a4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:34.676828 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fc8ca-54ab-4aee-ab95-8fad308718a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:34.678183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fc8ca-54ab-4aee-ab95-8fad308718a4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:34.683345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 441/880] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Aug 4 07:24:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:34.688630 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ab905-0980-4204-b4a3-95a30b737fc7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:34.800393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ab905-0980-4204-b4a3-95a30b737fc7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:34.837093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb36096-7ea8-40a4-9b62-49ee21863ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:34.837714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb36096-7ea8-40a4-9b62-49ee21863ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:34.837997 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb36096-7ea8-40a4-9b62-49ee21863ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:34.838268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb36096-7ea8-40a4-9b62-49ee21863ce5 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:34.838949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb36096-7ea8-40a4-9b62-49ee21863ce5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:34.839248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb36096-7ea8-40a4-9b62-49ee21863ce5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:34.840699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb36096-7ea8-40a4-9b62-49ee21863ce5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:34.847730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 440/881] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Aug 4 07:24:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:34.856643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fc8ca-54ab-4aee-ab95-8fad308718a4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:34.922087 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:34.963785 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fc8ca-54ab-4aee-ab95-8fad308718a4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:34.981496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803dc37-be50-41b9-8ea9-0576da2d1fdc None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:34.981496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803dc37-be50-41b9-8ea9-0576da2d1fdc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:34.981496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803dc37-be50-41b9-8ea9-0576da2d1fdc None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:34.981496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 441/882] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:34.991120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e1ae5-0a80-48ca-bdcc-72fd5d0d4471 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:34.991120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e1ae5-0a80-48ca-bdcc-72fd5d0d4471 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:34.991120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e1ae5-0a80-48ca-bdcc-72fd5d0d4471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:35.003677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfce3fc-4051-4bf9-9641-09a4e38aa48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:35.004711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce3fc-4051-4bf9-9641-09a4e38aa48a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:35.005219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce3fc-4051-4bf9-9641-09a4e38aa48a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:35.005701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce3fc-4051-4bf9-9641-09a4e38aa48a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:35.006780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfce3fc-4051-4bf9-9641-09a4e38aa48a None admin] RBAC: Adding query filter params (interface=public, service_id=16edd966115b4ec0aa85c2c5f91710b8, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:35.007166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-740e1ae5-0a80-48ca-bdcc-72fd5d0d4471 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:35.007527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfce3fc-4051-4bf9-9641-09a4e38aa48a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:35.008391 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-740e1ae5-0a80-48ca-bdcc-72fd5d0d4471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:35.009368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfce3fc-4051-4bf9-9641-09a4e38aa48a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:35.015403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 442/883] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Aug 4 07:24:34 2022] GET /identity/v3/endpoints?service_id=16edd966115b4ec0aa85c2c5f91710b8&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:35.021924 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfce3fc-4051-4bf9-9641-09a4e38aa48a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:35.043952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:35.100157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 442/884] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:35.120844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e1ae5-0a80-48ca-bdcc-72fd5d0d4471 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:35.145701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfce3fc-4051-4bf9-9641-09a4e38aa48a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:35.173596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:35.182741 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab797e-3f51-4ca1-823a-e1697131fe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:35.186424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab797e-3f51-4ca1-823a-e1697131fe8b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:35.186424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab797e-3f51-4ca1-823a-e1697131fe8b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:35.186424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab797e-3f51-4ca1-823a-e1697131fe8b None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:35.186424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab797e-3f51-4ca1-823a-e1697131fe8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:35.186424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab797e-3f51-4ca1-823a-e1697131fe8b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:35.193245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 443/885] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Aug 4 07:24:35 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:24:35.239834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e1ae5-0a80-48ca-bdcc-72fd5d0d4471 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:35.279851 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25090666-ae39-461f-bff8-592937a5a7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:35.280853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25090666-ae39-461f-bff8-592937a5a7ae None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:35.281350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25090666-ae39-461f-bff8-592937a5a7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:35.281829 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25090666-ae39-461f-bff8-592937a5a7ae None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:35.282657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25090666-ae39-461f-bff8-592937a5a7ae None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:35.285080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25090666-ae39-461f-bff8-592937a5a7ae None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-25090666-ae39-461f-bff8-592937a5a7ae None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:35.295514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:35.296644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 07:24:35.297538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:35.300368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 443/886] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:24:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:35.306675 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab797e-3f51-4ca1-823a-e1697131fe8b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:35.417811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab797e-3f51-4ca1-823a-e1697131fe8b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:35.453905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c6b50-dcc0-41a0-90b5-14aea6bb3b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:35.455344 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c6b50-dcc0-41a0-90b5-14aea6bb3b23 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:35.456030 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c6b50-dcc0-41a0-90b5-14aea6bb3b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:35.456463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c6b50-dcc0-41a0-90b5-14aea6bb3b23 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:35.457301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c6b50-dcc0-41a0-90b5-14aea6bb3b23 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:35.457871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c6b50-dcc0-41a0-90b5-14aea6bb3b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:35.459397 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c6b50-dcc0-41a0-90b5-14aea6bb3b23 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:35.465126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 444/887] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:24:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:35.471494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25090666-ae39-461f-bff8-592937a5a7ae None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:35.502337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:35.586580 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25090666-ae39-461f-bff8-592937a5a7ae None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:35.623137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b504c38-eac4-43bc-91fb-8a08fcf93354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:35.623754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504c38-eac4-43bc-91fb-8a08fcf93354 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:35.624065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504c38-eac4-43bc-91fb-8a08fcf93354 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:35.624344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504c38-eac4-43bc-91fb-8a08fcf93354 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:35.625023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b504c38-eac4-43bc-91fb-8a08fcf93354 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:35.625329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b504c38-eac4-43bc-91fb-8a08fcf93354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:35.626752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b504c38-eac4-43bc-91fb-8a08fcf93354 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:35.632101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 444/888] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:24:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:35.637913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c6b50-dcc0-41a0-90b5-14aea6bb3b23 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:35.699403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:35.746739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c6b50-dcc0-41a0-90b5-14aea6bb3b23 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:35.781365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926d608-72db-4b79-9cf6-380e8a2c9b2e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:35.781657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926d608-72db-4b79-9cf6-380e8a2c9b2e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:35.781942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926d608-72db-4b79-9cf6-380e8a2c9b2e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:35.784147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 445/889] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:24:35.784346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444970c4-eadd-434f-8ade-f2ee11505204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:35.784948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444970c4-eadd-434f-8ade-f2ee11505204 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:35.785285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444970c4-eadd-434f-8ade-f2ee11505204 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:35.785572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444970c4-eadd-434f-8ade-f2ee11505204 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:35.786506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444970c4-eadd-434f-8ade-f2ee11505204 None admin] RBAC: Adding query filter params (interface=public, service_id=f8bb1391ec9e44ceb83875891ae5331e, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:35.786821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444970c4-eadd-434f-8ade-f2ee11505204 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:35.788255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444970c4-eadd-434f-8ade-f2ee11505204 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:35.790760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe182b98-7d06-4abc-80b1-d53b7c3e7ab5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:35.791273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe182b98-7d06-4abc-80b1-d53b7c3e7ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:35.791748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe182b98-7d06-4abc-80b1-d53b7c3e7ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:35.794104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 445/890] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Aug 4 07:24:35 2022] GET /identity/v3/endpoints?service_id=f8bb1391ec9e44ceb83875891ae5331e&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:35.799406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444970c4-eadd-434f-8ade-f2ee11505204 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:35.813449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fe182b98-7d06-4abc-80b1-d53b7c3e7ab5 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:35.815109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fe182b98-7d06-4abc-80b1-d53b7c3e7ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:35.908711 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 446/891] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:35.911957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444970c4-eadd-434f-8ade-f2ee11505204 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:35.927709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe182b98-7d06-4abc-80b1-d53b7c3e7ab5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:35.931913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:35.955245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:35.958561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cae69a-9bab-4821-b989-f6ac9d75f91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:35.959468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cae69a-9bab-4821-b989-f6ac9d75f91b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:35.959923 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cae69a-9bab-4821-b989-f6ac9d75f91b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:35.960803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cae69a-9bab-4821-b989-f6ac9d75f91b None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:35.961732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cae69a-9bab-4821-b989-f6ac9d75f91b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:35.963370 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cae69a-9bab-4821-b989-f6ac9d75f91b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:35.971252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 446/892] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Aug 4 07:24:35 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:24:36.048218 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe182b98-7d06-4abc-80b1-d53b7c3e7ab5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:36.086882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4fbe0-6571-4fd1-8b19-a41d37488037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:36.087546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4fbe0-6571-4fd1-8b19-a41d37488037 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:36.087831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4fbe0-6571-4fd1-8b19-a41d37488037 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:36.088142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4fbe0-6571-4fd1-8b19-a41d37488037 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:36.100645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4fbe0-6571-4fd1-8b19-a41d37488037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e1a4fbe0-6571-4fd1-8b19-a41d37488037 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:36.105776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:36.107263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:36.108146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 447/893] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Aug 4 07:24:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:36.114434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cae69a-9bab-4821-b989-f6ac9d75f91b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:36.225241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cae69a-9bab-4821-b989-f6ac9d75f91b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:36.261670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1796d7f-9d95-4634-974a-bffd0994168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:36.262684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1796d7f-9d95-4634-974a-bffd0994168e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:36.265704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1796d7f-9d95-4634-974a-bffd0994168e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:36.265704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1796d7f-9d95-4634-974a-bffd0994168e None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:36.265704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1796d7f-9d95-4634-974a-bffd0994168e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:36.265704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1796d7f-9d95-4634-974a-bffd0994168e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:36.265983 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1796d7f-9d95-4634-974a-bffd0994168e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:36.275718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 447/894] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Aug 4 07:24:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:36.281492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4fbe0-6571-4fd1-8b19-a41d37488037 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:36.367534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:36.385487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4fbe0-6571-4fd1-8b19-a41d37488037 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:36.421227 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4c3c5-6d7e-4cc5-a793-b12509e3e152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:36.421871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4c3c5-6d7e-4cc5-a793-b12509e3e152 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:36.422154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4c3c5-6d7e-4cc5-a793-b12509e3e152 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:36.422428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4c3c5-6d7e-4cc5-a793-b12509e3e152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:36.425091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dd831-0008-4462-8974-e6874031ea27 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:36.425408 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dd831-0008-4462-8974-e6874031ea27 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:36.425686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dd831-0008-4462-8974-e6874031ea27 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:36.430416 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 448/895] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:24:36.432499 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83ad58-7027-4202-bec8-8071bfade6e6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:36.432786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83ad58-7027-4202-bec8-8071bfade6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:36.433067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83ad58-7027-4202-bec8-8071bfade6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:36.435026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4c3c5-6d7e-4cc5-a793-b12509e3e152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:36.436834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4c3c5-6d7e-4cc5-a793-b12509e3e152 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-38c4c3c5-6d7e-4cc5-a793-b12509e3e152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:36.453902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:36.455445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:36.457304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:36.457304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:36.457304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:36.457304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:36.457304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 448/896] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:36.461852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6e83ad58-7027-4202-bec8-8071bfade6e6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:36.462033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4c3c5-6d7e-4cc5-a793-b12509e3e152 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:36.463590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6e83ad58-7027-4202-bec8-8071bfade6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:36.516454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:36.539970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 449/897] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:36.562479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83ad58-7027-4202-bec8-8071bfade6e6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:36.567959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4c3c5-6d7e-4cc5-a793-b12509e3e152 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:36.609488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1557ab-2afb-4a20-97c2-a2f506f35e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:36.610108 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1557ab-2afb-4a20-97c2-a2f506f35e0f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:36.610393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1557ab-2afb-4a20-97c2-a2f506f35e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:36.610673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1557ab-2afb-4a20-97c2-a2f506f35e0f None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:36.611486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1557ab-2afb-4a20-97c2-a2f506f35e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:36.611796 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1557ab-2afb-4a20-97c2-a2f506f35e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:36.613686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1557ab-2afb-4a20-97c2-a2f506f35e0f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:36.615759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:36.631681 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 449/898] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:36.636875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1557ab-2afb-4a20-97c2-a2f506f35e0f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:36.689951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83ad58-7027-4202-bec8-8071bfade6e6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:36.726460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fca36c-1e72-450c-8062-c92bc14086a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:36.727542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fca36c-1e72-450c-8062-c92bc14086a2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:36.727976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fca36c-1e72-450c-8062-c92bc14086a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:36.728505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fca36c-1e72-450c-8062-c92bc14086a2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:36.729301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fca36c-1e72-450c-8062-c92bc14086a2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:36.731039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fca36c-1e72-450c-8062-c92bc14086a2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c8fca36c-1e72-450c-8062-c92bc14086a2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:36.745799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:36.747026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:36.748181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:36.749894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 450/899] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Aug 4 07:24:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:36.756577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fca36c-1e72-450c-8062-c92bc14086a2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:36.773188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1557ab-2afb-4a20-97c2-a2f506f35e0f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:36.803951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:36.813730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e592d-6ab0-45bf-b0eb-abc55f47d7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:36.815448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e592d-6ab0-45bf-b0eb-abc55f47d7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:36.815791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e592d-6ab0-45bf-b0eb-abc55f47d7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:36.816174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e592d-6ab0-45bf-b0eb-abc55f47d7f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:36.826249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e592d-6ab0-45bf-b0eb-abc55f47d7f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:36.828271 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e592d-6ab0-45bf-b0eb-abc55f47d7f7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-311e592d-6ab0-45bf-b0eb-abc55f47d7f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:36.843495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:36.844699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:36.846550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:36.846550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:36.846550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:36.846550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:36.846550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:36.846550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:36.846550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:36.846550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:36.846550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:36.846550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:36.846550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 450/900] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Aug 4 07:24:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:36.853267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e592d-6ab0-45bf-b0eb-abc55f47d7f7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:36.878745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fca36c-1e72-450c-8062-c92bc14086a2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:36.925193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93bfa7-44dd-4d2d-b416-a219980123c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:36.926226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93bfa7-44dd-4d2d-b416-a219980123c5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:36.926866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:36.927337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93bfa7-44dd-4d2d-b416-a219980123c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:36.928580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93bfa7-44dd-4d2d-b416-a219980123c5 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:36.929725 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93bfa7-44dd-4d2d-b416-a219980123c5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:36.930384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93bfa7-44dd-4d2d-b416-a219980123c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:36.932636 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93bfa7-44dd-4d2d-b416-a219980123c5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:36.939773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 451/901] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Aug 4 07:24:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:36.947446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93bfa7-44dd-4d2d-b416-a219980123c5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:36.986117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e592d-6ab0-45bf-b0eb-abc55f47d7f7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:37.030913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a536be2-6678-44ec-990f-78eee8f54d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:37.031601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a536be2-6678-44ec-990f-78eee8f54d32 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:37.031889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a536be2-6678-44ec-990f-78eee8f54d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:37.032195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a536be2-6678-44ec-990f-78eee8f54d32 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:37.033025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a536be2-6678-44ec-990f-78eee8f54d32 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:37.033336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a536be2-6678-44ec-990f-78eee8f54d32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:37.035258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a536be2-6678-44ec-990f-78eee8f54d32 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:37.050015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93bfa7-44dd-4d2d-b416-a219980123c5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:37.052801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:37.060049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 451/902] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Aug 4 07:24:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:37.062399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a536be2-6678-44ec-990f-78eee8f54d32 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:37.090973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef562178-6e1f-4dd3-a2c4-1e32db0b6788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:37.091939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef562178-6e1f-4dd3-a2c4-1e32db0b6788 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:37.092498 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef562178-6e1f-4dd3-a2c4-1e32db0b6788 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:37.093023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef562178-6e1f-4dd3-a2c4-1e32db0b6788 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:37.093945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef562178-6e1f-4dd3-a2c4-1e32db0b6788 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:37.094454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef562178-6e1f-4dd3-a2c4-1e32db0b6788 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:37.096286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef562178-6e1f-4dd3-a2c4-1e32db0b6788 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:37.108157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 452/903] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Aug 4 07:24:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:37.183254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a536be2-6678-44ec-990f-78eee8f54d32 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:37.200062 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:37.225962 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebde7fd-fcf2-41b1-9b48-2121dbc3d41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:37.226589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebde7fd-fcf2-41b1-9b48-2121dbc3d41f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:37.226872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebde7fd-fcf2-41b1-9b48-2121dbc3d41f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:37.227220 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebde7fd-fcf2-41b1-9b48-2121dbc3d41f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:37.306583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebde7fd-fcf2-41b1-9b48-2121dbc3d41f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:37.307953 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebde7fd-fcf2-41b1-9b48-2121dbc3d41f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:37.412783 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 452/904] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:37 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 07:24:37.418310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef562178-6e1f-4dd3-a2c4-1e32db0b6788 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:37.435547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d444d-0253-4edf-9ea2-019da1d316c2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:37.435853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d444d-0253-4edf-9ea2-019da1d316c2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:37.436155 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d444d-0253-4edf-9ea2-019da1d316c2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:37.437763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 453/905] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:24:37.443529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08975500-2961-4781-a649-ab33153a0941 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:37.443819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08975500-2961-4781-a649-ab33153a0941 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:37.444136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08975500-2961-4781-a649-ab33153a0941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:37.462846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-08975500-2961-4781-a649-ab33153a0941 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:37.464342 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-08975500-2961-4781-a649-ab33153a0941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:37.472411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:37.530759 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef562178-6e1f-4dd3-a2c4-1e32db0b6788 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:37.538024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 454/906] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:37.560156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08975500-2961-4781-a649-ab33153a0941 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:37.567304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f78c1-5dc1-4e90-bfc7-bf56e7e1e195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:37.567954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f78c1-5dc1-4e90-bfc7-bf56e7e1e195 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:37.568259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f78c1-5dc1-4e90-bfc7-bf56e7e1e195 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:37.568591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f78c1-5dc1-4e90-bfc7-bf56e7e1e195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:37.580441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f78c1-5dc1-4e90-bfc7-bf56e7e1e195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:37.581901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f78c1-5dc1-4e90-bfc7-bf56e7e1e195 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8f0f78c1-5dc1-4e90-bfc7-bf56e7e1e195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:37.597897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:37.598808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:37.600321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:37.600321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:37.600321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:37.600321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:37.600321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 453/907] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:37.604677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f78c1-5dc1-4e90-bfc7-bf56e7e1e195 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:37.624782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:37.670257 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08975500-2961-4781-a649-ab33153a0941 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:37.718172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a0849-08da-4fbe-84ec-25b84f26fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:37.718942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a0849-08da-4fbe-84ec-25b84f26fe0e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:37.719549 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a0849-08da-4fbe-84ec-25b84f26fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:37.719902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a0849-08da-4fbe-84ec-25b84f26fe0e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:37.720702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a0849-08da-4fbe-84ec-25b84f26fe0e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:37.723010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a0849-08da-4fbe-84ec-25b84f26fe0e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:37.731898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f78c1-5dc1-4e90-bfc7-bf56e7e1e195 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a98a0849-08da-4fbe-84ec-25b84f26fe0e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:37.736166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:37.737140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 07:24:37.737955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:37.742339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 455/908] 10.222.0.64 () {56 vars in 1132 bytes} [Thu Aug 4 07:24:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:37.743228 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a0849-08da-4fbe-84ec-25b84f26fe0e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:37.756374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:37.775441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12474a-b965-4ed6-a9cb-3b71b4bad492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:37.776473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12474a-b965-4ed6-a9cb-3b71b4bad492 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:37.776956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12474a-b965-4ed6-a9cb-3b71b4bad492 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:37.777426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12474a-b965-4ed6-a9cb-3b71b4bad492 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:37.778311 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12474a-b965-4ed6-a9cb-3b71b4bad492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:37.778963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12474a-b965-4ed6-a9cb-3b71b4bad492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:37.780744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12474a-b965-4ed6-a9cb-3b71b4bad492 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:37.794625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 454/909] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:37.801409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12474a-b965-4ed6-a9cb-3b71b4bad492 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:37.867011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a0849-08da-4fbe-84ec-25b84f26fe0e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:37.905289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e3f82-3b9a-47e8-9a45-3a0568cc9c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:37.906383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e3f82-3b9a-47e8-9a45-3a0568cc9c3f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:37.906863 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e3f82-3b9a-47e8-9a45-3a0568cc9c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:37.907669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:37.907941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e3f82-3b9a-47e8-9a45-3a0568cc9c3f None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:37.909481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e3f82-3b9a-47e8-9a45-3a0568cc9c3f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:37.910042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e3f82-3b9a-47e8-9a45-3a0568cc9c3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:37.911856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e3f82-3b9a-47e8-9a45-3a0568cc9c3f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:37.918340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 456/910] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Aug 4 07:24:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:37.923791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e3f82-3b9a-47e8-9a45-3a0568cc9c3f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:37.932309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12474a-b965-4ed6-a9cb-3b71b4bad492 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:37.980221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fb9fa-2e55-4202-9db6-bf1fa7fe6cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:37.980867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fb9fa-2e55-4202-9db6-bf1fa7fe6cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:37.981151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fb9fa-2e55-4202-9db6-bf1fa7fe6cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:37.981431 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fb9fa-2e55-4202-9db6-bf1fa7fe6cc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:37.991680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795fb9fa-2e55-4202-9db6-bf1fa7fe6cc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:37.993564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795fb9fa-2e55-4202-9db6-bf1fa7fe6cc6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-795fb9fa-2e55-4202-9db6-bf1fa7fe6cc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:38.009623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:38.010551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:38.011472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:38.011472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:38.011472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:38.011472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:38.011472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:38.011472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:38.011472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:38.011472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:38.016181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 455/911] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:38.017035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fb9fa-2e55-4202-9db6-bf1fa7fe6cc6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:38.019623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:38.054137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e3f82-3b9a-47e8-9a45-3a0568cc9c3f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:38.099832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76f8d7-2e68-427b-ac4a-e8d6d9bcb3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:38.100515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76f8d7-2e68-427b-ac4a-e8d6d9bcb3cf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:38.100817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76f8d7-2e68-427b-ac4a-e8d6d9bcb3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:38.101096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76f8d7-2e68-427b-ac4a-e8d6d9bcb3cf None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:38.101827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76f8d7-2e68-427b-ac4a-e8d6d9bcb3cf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:38.102138 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76f8d7-2e68-427b-ac4a-e8d6d9bcb3cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:38.103802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76f8d7-2e68-427b-ac4a-e8d6d9bcb3cf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:38.112023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 457/912] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Aug 4 07:24:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:38.119364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76f8d7-2e68-427b-ac4a-e8d6d9bcb3cf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:38.145962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fb9fa-2e55-4202-9db6-bf1fa7fe6cc6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:38.185050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ec3f2-e4f6-4169-97fe-93b1ec5e7eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:38.186222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ec3f2-e4f6-4169-97fe-93b1ec5e7eed None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:38.186937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ec3f2-e4f6-4169-97fe-93b1ec5e7eed None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:38.187822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ec3f2-e4f6-4169-97fe-93b1ec5e7eed None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:38.189638 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ec3f2-e4f6-4169-97fe-93b1ec5e7eed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:38.190157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ec3f2-e4f6-4169-97fe-93b1ec5e7eed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:38.192553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ec3f2-e4f6-4169-97fe-93b1ec5e7eed None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:38.202679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:38.214403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 456/913] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:38.214403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50d76b-7b20-4c3e-a2ec-6d75f631472c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:38.214403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50d76b-7b20-4c3e-a2ec-6d75f631472c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:38.214403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50d76b-7b20-4c3e-a2ec-6d75f631472c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:38.215153 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 457/914] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:38.215932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50d76b-7b20-4c3e-a2ec-6d75f631472c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:38.232855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76f8d7-2e68-427b-ac4a-e8d6d9bcb3cf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:38.271738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f4ef9-6f96-4848-a9c3-d0df42694992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:38.272703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f4ef9-6f96-4848-a9c3-d0df42694992 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:38.273603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f4ef9-6f96-4848-a9c3-d0df42694992 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:38.274065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f4ef9-6f96-4848-a9c3-d0df42694992 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:38.274947 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f4ef9-6f96-4848-a9c3-d0df42694992 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:38.276592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f4ef9-6f96-4848-a9c3-d0df42694992 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:38.281242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:38.323029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50d76b-7b20-4c3e-a2ec-6d75f631472c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:38.355303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 458/915] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:24:38 2022] POST /identity/v3/endpoints => generated 347 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:24:38.357155 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276028a-36b9-43e2-bfd7-0165eaefcda6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:38.357872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276028a-36b9-43e2-bfd7-0165eaefcda6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:38.358308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276028a-36b9-43e2-bfd7-0165eaefcda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:38.378685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95eb84-8a47-43f0-86c7-3b8e49901cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:38.379290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95eb84-8a47-43f0-86c7-3b8e49901cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:38.379578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95eb84-8a47-43f0-86c7-3b8e49901cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:38.379853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95eb84-8a47-43f0-86c7-3b8e49901cd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:38.380722 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95eb84-8a47-43f0-86c7-3b8e49901cd8 None admin] RBAC: Adding query filter params (role.id=eefaec6a74bd4d5c93eca0ae44a177fa, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=0c30c2fd1c4648308fac6606526451ec) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:38.381037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95eb84-8a47-43f0-86c7-3b8e49901cd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:38.381298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8276028a-36b9-43e2-bfd7-0165eaefcda6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:38.382569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95eb84-8a47-43f0-86c7-3b8e49901cd8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:38.383255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8276028a-36b9-43e2-bfd7-0165eaefcda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:38.396249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 458/916] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:38 2022] GET /identity/v3/role_assignments?user.id=0c30c2fd1c4648308fac6606526451ec&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=eefaec6a74bd4d5c93eca0ae44a177fa => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:38.474278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 459/917] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:38.498103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95eb84-8a47-43f0-86c7-3b8e49901cd8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:38.568829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:38.614737 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95eb84-8a47-43f0-86c7-3b8e49901cd8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:38.652710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8eb4a-c53c-4528-a8bb-0ceb38cb0b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:38.653763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8eb4a-c53c-4528-a8bb-0ceb38cb0b93 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:38.654257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8eb4a-c53c-4528-a8bb-0ceb38cb0b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:38.654751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8eb4a-c53c-4528-a8bb-0ceb38cb0b93 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:38.655591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8eb4a-c53c-4528-a8bb-0ceb38cb0b93 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:38.657224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8eb4a-c53c-4528-a8bb-0ceb38cb0b93 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cab8eb4a-c53c-4528-a8bb-0ceb38cb0b93 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:38.667534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:38.668570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 07:24:38.669489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:38.672321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 459/918] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:24:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:38.679065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276028a-36b9-43e2-bfd7-0165eaefcda6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:38.785869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276028a-36b9-43e2-bfd7-0165eaefcda6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:38.806870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:38.821570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce8ef7-ac19-4bd5-aa7d-76d622be0a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:38.822187 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8ef7-ac19-4bd5-aa7d-76d622be0a9a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:38.825605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8ef7-ac19-4bd5-aa7d-76d622be0a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:38.825605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8ef7-ac19-4bd5-aa7d-76d622be0a9a None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:38.825605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce8ef7-ac19-4bd5-aa7d-76d622be0a9a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:38.825605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce8ef7-ac19-4bd5-aa7d-76d622be0a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:38.825605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce8ef7-ac19-4bd5-aa7d-76d622be0a9a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:38.830977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 460/919] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:24:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:38.836571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8eb4a-c53c-4528-a8bb-0ceb38cb0b93 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:38.951955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8eb4a-c53c-4528-a8bb-0ceb38cb0b93 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:38.980890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:38.998072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce56a40-d7fa-40c9-8ea6-1092cd002eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:39.003671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce56a40-d7fa-40c9-8ea6-1092cd002eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:39.003671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce56a40-d7fa-40c9-8ea6-1092cd002eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:39.003671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce56a40-d7fa-40c9-8ea6-1092cd002eb0 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:39.003671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce56a40-d7fa-40c9-8ea6-1092cd002eb0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:39.003671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce56a40-d7fa-40c9-8ea6-1092cd002eb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:39.003671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce56a40-d7fa-40c9-8ea6-1092cd002eb0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:39.008347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 460/920] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Aug 4 07:24:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:39.014176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce8ef7-ac19-4bd5-aa7d-76d622be0a9a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:39.121414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce8ef7-ac19-4bd5-aa7d-76d622be0a9a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:39.158148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1beced-847a-4aca-9233-7966ceb73b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:39.159143 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1beced-847a-4aca-9233-7966ceb73b9d None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:39.159619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1beced-847a-4aca-9233-7966ceb73b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:39.160100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1beced-847a-4aca-9233-7966ceb73b9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:39.161010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1beced-847a-4aca-9233-7966ceb73b9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:39.162566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1beced-847a-4aca-9233-7966ceb73b9d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:39.269496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aef0a-d0d0-441c-bfe9-9f78fa50ccde None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:39.269735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aef0a-d0d0-441c-bfe9-9f78fa50ccde None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:39.269993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aef0a-d0d0-441c-bfe9-9f78fa50ccde None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:39.271531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 461/921] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:24:39.276186 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b04565-3e08-40dc-b97f-223b2f3925bf None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:39.276479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b04565-3e08-40dc-b97f-223b2f3925bf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:39.276762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b04565-3e08-40dc-b97f-223b2f3925bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:39.295244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-89b04565-3e08-40dc-b97f-223b2f3925bf None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:39.296459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-89b04565-3e08-40dc-b97f-223b2f3925bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:39.303048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 461/922] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:24:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:24:39.385621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 462/923] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:39.411912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1beced-847a-4aca-9233-7966ceb73b9d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:39.483266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:39.557005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1beced-847a-4aca-9233-7966ceb73b9d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:39.597711 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe08fec-afde-4871-9b00-6d6cab2ecf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:39.598739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe08fec-afde-4871-9b00-6d6cab2ecf4c None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:39.599311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe08fec-afde-4871-9b00-6d6cab2ecf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:39.599799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe08fec-afde-4871-9b00-6d6cab2ecf4c None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:39.600727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe08fec-afde-4871-9b00-6d6cab2ecf4c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:39.602317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe08fec-afde-4871-9b00-6d6cab2ecf4c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:39.689385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 462/924] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:24:39 2022] POST /identity/v3/services => generated 241 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:40.502589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7574b0-667f-46bd-be00-39f0549b33ad None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:40.503109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7574b0-667f-46bd-be00-39f0549b33ad None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:40.503186 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7574b0-667f-46bd-be00-39f0549b33ad None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:40.504898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 463/925] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:40.510254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be6c97-cefc-4c20-b26f-aad6a46df844 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:40.510657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be6c97-cefc-4c20-b26f-aad6a46df844 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:40.510843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be6c97-cefc-4c20-b26f-aad6a46df844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:40.529735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-34be6c97-cefc-4c20-b26f-aad6a46df844 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:40.530925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-34be6c97-cefc-4c20-b26f-aad6a46df844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:40.612105 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 463/926] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:40.633224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7574b0-667f-46bd-be00-39f0549b33ad None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:40.681274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a347423-e169-4315-8c1d-78f89f52dc9f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:40.681563 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a347423-e169-4315-8c1d-78f89f52dc9f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:40.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a347423-e169-4315-8c1d-78f89f52dc9f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:40.684082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 464/927] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:24:40.691709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbbdd1-ab8d-468c-a848-d7bbb15cfde4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:40.692053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbbdd1-ab8d-468c-a848-d7bbb15cfde4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:40.692287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbbdd1-ab8d-468c-a848-d7bbb15cfde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:40.715290 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6bcbbdd1-ab8d-468c-a848-d7bbb15cfde4 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:40.716636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6bcbbdd1-ab8d-468c-a848-d7bbb15cfde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:40.752740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:40.753176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7574b0-667f-46bd-be00-39f0549b33ad None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:40.771743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:40.798651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce81986-c249-411c-9d30-835fe7a9e3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:40.799314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce81986-c249-411c-9d30-835fe7a9e3de None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:40.800011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce81986-c249-411c-9d30-835fe7a9e3de None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:40.800439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce81986-c249-411c-9d30-835fe7a9e3de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:40.812161 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 465/928] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:40.817117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce81986-c249-411c-9d30-835fe7a9e3de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0ce81986-c249-411c-9d30-835fe7a9e3de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:40.822283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:40.826070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:40.828234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 464/929] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Aug 4 07:24:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:40.830044 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbbdd1-ab8d-468c-a848-d7bbb15cfde4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:40.834466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce81986-c249-411c-9d30-835fe7a9e3de None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:40.952154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbbdd1-ab8d-468c-a848-d7bbb15cfde4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:40.989643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce81986-c249-411c-9d30-835fe7a9e3de None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:40.999610 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e191f09-0c05-4f9c-8820-752e7e1cbfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:41.001000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e191f09-0c05-4f9c-8820-752e7e1cbfb4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:41.002219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e191f09-0c05-4f9c-8820-752e7e1cbfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:41.002689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e191f09-0c05-4f9c-8820-752e7e1cbfb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:41.003642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e191f09-0c05-4f9c-8820-752e7e1cbfb4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:41.004151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e191f09-0c05-4f9c-8820-752e7e1cbfb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:41.005922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e191f09-0c05-4f9c-8820-752e7e1cbfb4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:41.021438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 466/930] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Aug 4 07:24:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:41.023104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e191f09-0c05-4f9c-8820-752e7e1cbfb4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:41.036622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78b87d-7dc6-4ff3-946e-d9d1951ffff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:41.037356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78b87d-7dc6-4ff3-946e-d9d1951ffff0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:41.037640 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78b87d-7dc6-4ff3-946e-d9d1951ffff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:41.037914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78b87d-7dc6-4ff3-946e-d9d1951ffff0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:41.038548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78b87d-7dc6-4ff3-946e-d9d1951ffff0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:41.040368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78b87d-7dc6-4ff3-946e-d9d1951ffff0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:41.048396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1c78b87d-7dc6-4ff3-946e-d9d1951ffff0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:41.051875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:41.053551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:41.054749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 465/931] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:41.060687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78b87d-7dc6-4ff3-946e-d9d1951ffff0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:41.107727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:41.153562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e191f09-0c05-4f9c-8820-752e7e1cbfb4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:41.189586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78b87d-7dc6-4ff3-946e-d9d1951ffff0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:41.200894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993992a-c357-4c18-8ae2-63e3956a1cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:41.201549 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993992a-c357-4c18-8ae2-63e3956a1cdd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:41.201832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993992a-c357-4c18-8ae2-63e3956a1cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:41.202111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993992a-c357-4c18-8ae2-63e3956a1cdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:41.219083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993992a-c357-4c18-8ae2-63e3956a1cdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:41.222194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993992a-c357-4c18-8ae2-63e3956a1cdd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:41.231027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b2a71-6c68-4b79-a70c-cd7092bd0ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:41.232300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b2a71-6c68-4b79-a70c-cd7092bd0ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:41.232805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b2a71-6c68-4b79-a70c-cd7092bd0ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:41.233286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b2a71-6c68-4b79-a70c-cd7092bd0ae0 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:41.234183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b2a71-6c68-4b79-a70c-cd7092bd0ae0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:41.234871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b2a71-6c68-4b79-a70c-cd7092bd0ae0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:41.236639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b2a71-6c68-4b79-a70c-cd7092bd0ae0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1993992a-c357-4c18-8ae2-63e3956a1cdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:41.242306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:41.249929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:41.250840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:41.250840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:41.250840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:41.250840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:41.251061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 466/932] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:41.251222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 467/933] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:41.251355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b2a71-6c68-4b79-a70c-cd7092bd0ae0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:41.255385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993992a-c357-4c18-8ae2-63e3956a1cdd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:41.351624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:41.380238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:41.384931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b2a71-6c68-4b79-a70c-cd7092bd0ae0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:41.402593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993992a-c357-4c18-8ae2-63e3956a1cdd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:41.426853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a671f0-4bf9-4f8c-bd03-737c9d1b5cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:41.427515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a671f0-4bf9-4f8c-bd03-737c9d1b5cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:41.427799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a671f0-4bf9-4f8c-bd03-737c9d1b5cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:41.428126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a671f0-4bf9-4f8c-bd03-737c9d1b5cc2 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:41.428821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a671f0-4bf9-4f8c-bd03-737c9d1b5cc2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:41.429127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a671f0-4bf9-4f8c-bd03-737c9d1b5cc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:41.430578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a671f0-4bf9-4f8c-bd03-737c9d1b5cc2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:41.436033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 467/934] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:41.452553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b3de6-e534-4717-b6b0-f2da971d9b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:41.453528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b3de6-e534-4717-b6b0-f2da971d9b05 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:41.454026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b3de6-e534-4717-b6b0-f2da971d9b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:41.455091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b3de6-e534-4717-b6b0-f2da971d9b05 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:41.455967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b3de6-e534-4717-b6b0-f2da971d9b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:41.456447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b3de6-e534-4717-b6b0-f2da971d9b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:41.458231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b3de6-e534-4717-b6b0-f2da971d9b05 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:41.478784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 468/935] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:41.480576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a671f0-4bf9-4f8c-bd03-737c9d1b5cc2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:41.601029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a671f0-4bf9-4f8c-bd03-737c9d1b5cc2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:41.640137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f4169-47dc-4efd-ad3b-e6a4554c9a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:41.641050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f4169-47dc-4efd-ad3b-e6a4554c9a05 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:41.641482 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f4169-47dc-4efd-ad3b-e6a4554c9a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:41.642524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f4169-47dc-4efd-ad3b-e6a4554c9a05 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:41.650868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f4169-47dc-4efd-ad3b-e6a4554c9a05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:41.652650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f4169-47dc-4efd-ad3b-e6a4554c9a05 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:41.663392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d5e8b-d5ba-4e7a-acbc-5937edd09d77 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:41.664031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d5e8b-d5ba-4e7a-acbc-5937edd09d77 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:41.664031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d5e8b-d5ba-4e7a-acbc-5937edd09d77 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:41.665458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 469/936] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-463f4169-47dc-4efd-ad3b-e6a4554c9a05 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:41.669192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:41.671363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:41.673803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:41.673803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:41.673803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:41.673803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:41.673803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:41.673803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:41.673803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:41.673803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:41.673803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:41.673803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:41.675541 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 468/937] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Aug 4 07:24:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:41.677633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bd2eb-f478-41af-b7b3-a62226e2837a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:41.677633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bd2eb-f478-41af-b7b3-a62226e2837a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:41.681135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f4169-47dc-4efd-ad3b-e6a4554c9a05 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:41.681345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bd2eb-f478-41af-b7b3-a62226e2837a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:41.696973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-586bd2eb-f478-41af-b7b3-a62226e2837a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:41.698285 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-586bd2eb-f478-41af-b7b3-a62226e2837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:41.706879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:41.789480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f4169-47dc-4efd-ad3b-e6a4554c9a05 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:41.789882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 470/938] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:41.809469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bd2eb-f478-41af-b7b3-a62226e2837a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:41.826474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccbc5b-9913-44d9-8c16-81a0cc8f9592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:41.826969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccbc5b-9913-44d9-8c16-81a0cc8f9592 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:41.827253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccbc5b-9913-44d9-8c16-81a0cc8f9592 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:41.827529 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccbc5b-9913-44d9-8c16-81a0cc8f9592 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:41.828395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccbc5b-9913-44d9-8c16-81a0cc8f9592 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:41.828708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccbc5b-9913-44d9-8c16-81a0cc8f9592 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:41.830677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccbc5b-9913-44d9-8c16-81a0cc8f9592 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:41.855559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 469/939] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Aug 4 07:24:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:41.857253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccbc5b-9913-44d9-8c16-81a0cc8f9592 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:41.880232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:41.925435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bd2eb-f478-41af-b7b3-a62226e2837a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:41.950008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:41.964626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaaf198-51bc-40e7-ab60-dd4d8545576b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:41.965267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaf198-51bc-40e7-ab60-dd4d8545576b None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:41.965551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaf198-51bc-40e7-ab60-dd4d8545576b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:41.965829 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaf198-51bc-40e7-ab60-dd4d8545576b None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:41.966520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaaf198-51bc-40e7-ab60-dd4d8545576b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:41.967999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaaf198-51bc-40e7-ab60-dd4d8545576b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:41.971189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: WARNING keystone.server.flask.common [None req-1aaaf198-51bc-40e7-ab60-dd4d8545576b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 07:24:41.971967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccbc5b-9913-44d9-8c16-81a0cc8f9592 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:42.006352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5972bf9-8da7-4812-a163-b729626195e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:42.006989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5972bf9-8da7-4812-a163-b729626195e6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.007269 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5972bf9-8da7-4812-a163-b729626195e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.007544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5972bf9-8da7-4812-a163-b729626195e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.017994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5972bf9-8da7-4812-a163-b729626195e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:42.019953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5972bf9-8da7-4812-a163-b729626195e6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d5972bf9-8da7-4812-a163-b729626195e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:42.035168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:42.036111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:42.037032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:42.037032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:42.037032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:42.037032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:42.037032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 470/940] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:42.037578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f5e16-d350-4af5-824d-66c301dbdb27 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.037747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f5e16-d350-4af5-824d-66c301dbdb27 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.038018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f5e16-d350-4af5-824d-66c301dbdb27 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.039871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.sql.core [None req-1aaaf198-51bc-40e7-ab60-dd4d8545576b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 07:24:42.039871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 07:24:42.039871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [parameters: {'id': '4d871aea76434841b3b27ea23c0ce10b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 07:24:42.039871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80297) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 07:24:42.041043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 471/941] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:42.042072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f5e16-d350-4af5-824d-66c301dbdb27 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1aaaf198-51bc-40e7-ab60-dd4d8545576b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4d871aea76434841b3b27ea23c0ce10b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4d871aea76434841b3b27ea23c0ce10b. Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 07:24:42.047476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 04 07:24:42.048547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application yield resource Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise self.value Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application session.commit() Aug 04 07:24:42.049671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.flush() Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application compat.raise_( Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 07:24:42.050558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 07:24:42.051454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 07:24:42.052263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [parameters: {'id': '4d871aea76434841b3b27ea23c0ce10b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:42.053156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:42.053926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:42.054826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:42.054826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 04 07:24:42.054826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 04 07:24:42.054826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:42.054826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:42.054826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 04 07:24:42.054826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 04 07:24:42.054826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4d871aea76434841b3b27ea23c0ce10b. Aug 04 07:24:42.054826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:42.054826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 471/942] 10.222.0.64 () {60 vars in 1140 bytes} [Thu Aug 4 07:24:41 2022] POST /identity/v3/projects => generated 262 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:42.057056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189b32b-9cd7-4748-a3cb-d8c60e97fa2d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.057056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189b32b-9cd7-4748-a3cb-d8c60e97fa2d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.057056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189b32b-9cd7-4748-a3cb-d8c60e97fa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.076030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2189b32b-9cd7-4748-a3cb-d8c60e97fa2d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:42.077505 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2189b32b-9cd7-4748-a3cb-d8c60e97fa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:42.149406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 472/943] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:42.150151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189b32b-9cd7-4748-a3cb-d8c60e97fa2d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:42.163458 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f5e16-d350-4af5-824d-66c301dbdb27 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:42.200367 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:42.205184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf57f8-c33b-4364-8cca-c9aa1632574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:42.205799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf57f8-c33b-4364-8cca-c9aa1632574f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.206079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf57f8-c33b-4364-8cca-c9aa1632574f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.206353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf57f8-c33b-4364-8cca-c9aa1632574f None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.207041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf57f8-c33b-4364-8cca-c9aa1632574f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:42.207401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf57f8-c33b-4364-8cca-c9aa1632574f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:42.208797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf57f8-c33b-4364-8cca-c9aa1632574f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:42.210358 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:42.225259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 472/944] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:42.225259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf57f8-c33b-4364-8cca-c9aa1632574f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:42.261405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189b32b-9cd7-4748-a3cb-d8c60e97fa2d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:42.297200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d81f52-669d-4ed8-8502-2870f1794355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:42.297837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d81f52-669d-4ed8-8502-2870f1794355 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.298217 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d81f52-669d-4ed8-8502-2870f1794355 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.298492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d81f52-669d-4ed8-8502-2870f1794355 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.305097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d81f52-669d-4ed8-8502-2870f1794355 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:42.306748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d81f52-669d-4ed8-8502-2870f1794355 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-18d81f52-669d-4ed8-8502-2870f1794355 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:42.318246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:42.319107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:42.320025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:42.320025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:42.320025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:42.320025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:42.320025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:42.320025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:42.320025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:42.320025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:42.320025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 473/945] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:42.320519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d81f52-669d-4ed8-8502-2870f1794355 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:42.331714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf57f8-c33b-4364-8cca-c9aa1632574f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:42.372432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a08f2b-851b-4e1a-a00e-1c58e539d02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:42.373080 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a08f2b-851b-4e1a-a00e-1c58e539d02f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.373365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a08f2b-851b-4e1a-a00e-1c58e539d02f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.374002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a08f2b-851b-4e1a-a00e-1c58e539d02f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.374647 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a08f2b-851b-4e1a-a00e-1c58e539d02f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:42.376875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a08f2b-851b-4e1a-a00e-1c58e539d02f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f0a08f2b-851b-4e1a-a00e-1c58e539d02f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:42.386435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:42.387437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:42.388309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 473/946] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Aug 4 07:24:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:42.389119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a08f2b-851b-4e1a-a00e-1c58e539d02f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:42.439357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d81f52-669d-4ed8-8502-2870f1794355 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:42.447558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:42.451206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:42.474316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc13977-57dd-4677-94bb-80800154f014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:42.474957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc13977-57dd-4677-94bb-80800154f014 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.475239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc13977-57dd-4677-94bb-80800154f014 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.475521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc13977-57dd-4677-94bb-80800154f014 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.483330 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc13977-57dd-4677-94bb-80800154f014 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:42.484908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc13977-57dd-4677-94bb-80800154f014 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:42.494071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a08f2b-851b-4e1a-a00e-1c58e539d02f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fdc13977-57dd-4677-94bb-80800154f014 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:42.496603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:42.497476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:42.498386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:42.498386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:42.498386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:42.498386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:42.498386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:42.498386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:42.498386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:42.498386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:42.498386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 474/947] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:42.498865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc13977-57dd-4677-94bb-80800154f014 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:42.534961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f05ad-c7b1-4936-b01f-5b45cc0aa764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:42.535678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f05ad-c7b1-4936-b01f-5b45cc0aa764 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.536006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f05ad-c7b1-4936-b01f-5b45cc0aa764 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.536306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f05ad-c7b1-4936-b01f-5b45cc0aa764 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.537027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f05ad-c7b1-4936-b01f-5b45cc0aa764 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:42.537371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f05ad-c7b1-4936-b01f-5b45cc0aa764 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:42.538901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f05ad-c7b1-4936-b01f-5b45cc0aa764 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:42.567158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 474/948] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:42.568008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f05ad-c7b1-4936-b01f-5b45cc0aa764 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:42.610174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc13977-57dd-4677-94bb-80800154f014 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:42.653813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6d7e4-4abe-4ac0-bf13-ed3f51aa5bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:42.654345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d7e4-4abe-4ac0-bf13-ed3f51aa5bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.654631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d7e4-4abe-4ac0-bf13-ed3f51aa5bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.654905 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d7e4-4abe-4ac0-bf13-ed3f51aa5bd2 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.655693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6d7e4-4abe-4ac0-bf13-ed3f51aa5bd2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:42.655968 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6d7e4-4abe-4ac0-bf13-ed3f51aa5bd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:42.657584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6d7e4-4abe-4ac0-bf13-ed3f51aa5bd2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:42.666448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 475/949] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Aug 4 07:24:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:42.672342 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6d7e4-4abe-4ac0-bf13-ed3f51aa5bd2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:42.707456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:42.711394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f05ad-c7b1-4936-b01f-5b45cc0aa764 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:42.734993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:42.751815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cff166-a169-4c57-bcdc-6cfe2d3b579a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:42.752630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cff166-a169-4c57-bcdc-6cfe2d3b579a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.752989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cff166-a169-4c57-bcdc-6cfe2d3b579a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.753857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cff166-a169-4c57-bcdc-6cfe2d3b579a None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.754156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cff166-a169-4c57-bcdc-6cfe2d3b579a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:42.754551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cff166-a169-4c57-bcdc-6cfe2d3b579a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:42.756097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cff166-a169-4c57-bcdc-6cfe2d3b579a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:42.769085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 475/950] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:42.774732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cff166-a169-4c57-bcdc-6cfe2d3b579a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:42.793671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6d7e4-4abe-4ac0-bf13-ed3f51aa5bd2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:42.831581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7aff06-e03b-4856-aabe-1efdcf7d8deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:42.832225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7aff06-e03b-4856-aabe-1efdcf7d8deb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.832502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7aff06-e03b-4856-aabe-1efdcf7d8deb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.832769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7aff06-e03b-4856-aabe-1efdcf7d8deb None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.833437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7aff06-e03b-4856-aabe-1efdcf7d8deb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:42.833738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7aff06-e03b-4856-aabe-1efdcf7d8deb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:42.835097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7aff06-e03b-4856-aabe-1efdcf7d8deb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:42.840404 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 476/951] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Aug 4 07:24:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:42.846782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7aff06-e03b-4856-aabe-1efdcf7d8deb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:42.892341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cff166-a169-4c57-bcdc-6cfe2d3b579a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:42.936184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4956ef-e26a-403e-9d2c-7ca4ec53c564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:42.937193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4956ef-e26a-403e-9d2c-7ca4ec53c564 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:42.937661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4956ef-e26a-403e-9d2c-7ca4ec53c564 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:42.938126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4956ef-e26a-403e-9d2c-7ca4ec53c564 None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/0c30c2fd1c4648308fac6606526451ec/roles/eefaec6a74bd4d5c93eca0ae44a177fa` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:42.940868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4956ef-e26a-403e-9d2c-7ca4ec53c564 None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=0c30c2fd1c4648308fac6606526451ec, role_id=eefaec6a74bd4d5c93eca0ae44a177fa)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:42.943028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4956ef-e26a-403e-9d2c-7ca4ec53c564 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:42.967007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:42.971396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7aff06-e03b-4856-aabe-1efdcf7d8deb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:43.010097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341a55b-44e2-41e2-bb25-648afc0fb191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:43.010720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341a55b-44e2-41e2-bb25-648afc0fb191 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:43.011003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341a55b-44e2-41e2-bb25-648afc0fb191 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:43.011275 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341a55b-44e2-41e2-bb25-648afc0fb191 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:43.012092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341a55b-44e2-41e2-bb25-648afc0fb191 None admin] RBAC: Adding query filter params (interface=public, service_id=1a4ae4aa330d417fb375169e8a00a17b, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:43.012388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341a55b-44e2-41e2-bb25-648afc0fb191 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:43.013752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341a55b-44e2-41e2-bb25-648afc0fb191 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:43.022016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 477/952] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Aug 4 07:24:42 2022] GET /identity/v3/endpoints?service_id=1a4ae4aa330d417fb375169e8a00a17b&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:43.027535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341a55b-44e2-41e2-bb25-648afc0fb191 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:43.124200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 476/953] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:24:42 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/0c30c2fd1c4648308fac6606526451ec/roles/eefaec6a74bd4d5c93eca0ae44a177fa => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:24:43.132912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341a55b-44e2-41e2-bb25-648afc0fb191 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:43.176014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274e6d1-0746-4f92-9604-321268d9a247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:43.177088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274e6d1-0746-4f92-9604-321268d9a247 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:43.177581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274e6d1-0746-4f92-9604-321268d9a247 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:43.178066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274e6d1-0746-4f92-9604-321268d9a247 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:43.179024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274e6d1-0746-4f92-9604-321268d9a247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:43.180850 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274e6d1-0746-4f92-9604-321268d9a247 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:43.189332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 478/954] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Aug 4 07:24:43 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:24:43.551901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52df60-c47b-4442-a8e6-988f5d87322a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:43.552263 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52df60-c47b-4442-a8e6-988f5d87322a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:43.552617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52df60-c47b-4442-a8e6-988f5d87322a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:43.554247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 477/955] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:43.559537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9de92-21f6-476e-8e86-ff42d06f097b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:43.559828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9de92-21f6-476e-8e86-ff42d06f097b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:43.560135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9de92-21f6-476e-8e86-ff42d06f097b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:43.580247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2bb9de92-21f6-476e-8e86-ff42d06f097b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:43.581435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2bb9de92-21f6-476e-8e86-ff42d06f097b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:43.598825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:43.655015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 479/956] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:43.678900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52df60-c47b-4442-a8e6-988f5d87322a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:43.725441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:43.790083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52df60-c47b-4442-a8e6-988f5d87322a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:43.831144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef3de3-6588-4ad9-b788-e6d3e49c10de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:43.831687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef3de3-6588-4ad9-b788-e6d3e49c10de None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:43.831975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef3de3-6588-4ad9-b788-e6d3e49c10de None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:43.832281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef3de3-6588-4ad9-b788-e6d3e49c10de None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:43.833007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef3de3-6588-4ad9-b788-e6d3e49c10de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:43.834377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef3de3-6588-4ad9-b788-e6d3e49c10de None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:43.911082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 478/957] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:24:43 2022] POST /identity/v3/services => generated 242 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:44.814521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973123b-76a6-4915-9f56-42d978c0986a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:44.815049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973123b-76a6-4915-9f56-42d978c0986a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:44.815157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973123b-76a6-4915-9f56-42d978c0986a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:44.816792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 480/958] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:44.822033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6fb6-8b8b-4a9f-b09e-923c5d406389 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:44.822444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6fb6-8b8b-4a9f-b09e-923c5d406389 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:44.822613 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6fb6-8b8b-4a9f-b09e-923c5d406389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:44.841695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3d5b6fb6-8b8b-4a9f-b09e-923c5d406389 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:44.842870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3d5b6fb6-8b8b-4a9f-b09e-923c5d406389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:44.922512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 479/959] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:44.949583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973123b-76a6-4915-9f56-42d978c0986a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:45.062159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973123b-76a6-4915-9f56-42d978c0986a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:45.100792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33f64d-28d9-4142-b2c1-298d7b5a0d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:45.101325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33f64d-28d9-4142-b2c1-298d7b5a0d58 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:45.101609 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33f64d-28d9-4142-b2c1-298d7b5a0d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:45.101887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33f64d-28d9-4142-b2c1-298d7b5a0d58 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:45.102570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33f64d-28d9-4142-b2c1-298d7b5a0d58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:45.104035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33f64d-28d9-4142-b2c1-298d7b5a0d58 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:45.149917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.sql.core [None req-3f33f64d-28d9-4142-b2c1-298d7b5a0d58 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 07:24:45.149917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 07:24:45.149917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [parameters: {'user_id': '6a77a44b527244c39b579121fe92080a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 07:24:45.149917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80297) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3f33f64d-28d9-4142-b2c1-298d7b5a0d58 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Aug 04 07:24:45.156760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 07:24:45.158152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application yield resource Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise self.value Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application session.commit() Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 07:24:45.159142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.flush() Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application compat.raise_( Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 07:24:45.165608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 07:24:45.168259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 07:24:45.170320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [parameters: {'user_id': '6a77a44b527244c39b579121fe92080a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:45.172503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 04 07:24:45.176196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 04 07:24:45.178873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 04 07:24:45.178873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 04 07:24:45.178873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 07:24:45.178873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 07:24:45.178873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 07:24:45.178873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:45.178873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 481/960] 10.222.0.64 () {60 vars in 1131 bytes} [Thu Aug 4 07:24:44 2022] POST /identity/v3/users => generated 160 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:45.180751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b6fb6-8b8b-4a9f-b09e-923c5d406389 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:45.206672 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:45.220278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8956-2884-4d33-8dc5-7a19e1537600 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:45.220804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8956-2884-4d33-8dc5-7a19e1537600 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:45.221244 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8956-2884-4d33-8dc5-7a19e1537600 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:45.225943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 482/961] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:45.234214 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e679c34-4ccd-4490-a9fc-61f9692996a9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:45.234214 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e679c34-4ccd-4490-a9fc-61f9692996a9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:45.234214 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e679c34-4ccd-4490-a9fc-61f9692996a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:45.257583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7e679c34-4ccd-4490-a9fc-61f9692996a9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:45.259127 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7e679c34-4ccd-4490-a9fc-61f9692996a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:45.272205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:45.289332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b6fb6-8b8b-4a9f-b09e-923c5d406389 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:45.324895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05f984-8b95-466d-b84c-a2ac071d4ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:45.325832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05f984-8b95-466d-b84c-a2ac071d4ecd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:45.326211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05f984-8b95-466d-b84c-a2ac071d4ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:45.326564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05f984-8b95-466d-b84c-a2ac071d4ecd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:45.340011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 483/962] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:45.340011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72e2a7-b58a-4414-8c36-54c917085cd9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:45.340011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72e2a7-b58a-4414-8c36-54c917085cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:45.340011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72e2a7-b58a-4414-8c36-54c917085cd9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:45.340011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 484/963] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:45.343189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d32c70-a2ac-4179-a026-159d01a2d575 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:45.343280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05f984-8b95-466d-b84c-a2ac071d4ecd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:45.343387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d32c70-a2ac-4179-a026-159d01a2d575 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:45.343387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d32c70-a2ac-4179-a026-159d01a2d575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:45.343547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05f984-8b95-466d-b84c-a2ac071d4ecd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9a05f984-8b95-466d-b84c-a2ac071d4ecd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:45.358804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:45.359936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:45.361328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:45.361328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:45.361328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:45.361328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:45.361328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:45.361328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:45.361328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:45.361328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:45.361328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 07:24:45.361328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:45.362748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-89d32c70-a2ac-4179-a026-159d01a2d575 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:45.362916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 480/964] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:45.363072 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-89d32c70-a2ac-4179-a026-159d01a2d575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:45.363226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05f984-8b95-466d-b84c-a2ac071d4ecd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:45.433454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 485/965] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:45.434295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d32c70-a2ac-4179-a026-159d01a2d575 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:45.461537 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:45.477737 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05f984-8b95-466d-b84c-a2ac071d4ecd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:45.515531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e355d70-87a6-446a-94ab-f6e40cf13205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:45.516211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e355d70-87a6-446a-94ab-f6e40cf13205 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:45.516514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e355d70-87a6-446a-94ab-f6e40cf13205 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:45.516797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e355d70-87a6-446a-94ab-f6e40cf13205 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:45.524178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:45.528812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e355d70-87a6-446a-94ab-f6e40cf13205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6e355d70-87a6-446a-94ab-f6e40cf13205 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:45.534484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:45.540522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 481/966] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Aug 4 07:24:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:45.541367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e355d70-87a6-446a-94ab-f6e40cf13205 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:45.559887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d32c70-a2ac-4179-a026-159d01a2d575 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:45.596404 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a5bc8-7d08-40ba-9e48-4ff3952b2d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:45.597020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a5bc8-7d08-40ba-9e48-4ff3952b2d97 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:45.597301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a5bc8-7d08-40ba-9e48-4ff3952b2d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:45.597576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a5bc8-7d08-40ba-9e48-4ff3952b2d97 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:45.598331 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a5bc8-7d08-40ba-9e48-4ff3952b2d97 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:45.598635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a5bc8-7d08-40ba-9e48-4ff3952b2d97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:45.600233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a5bc8-7d08-40ba-9e48-4ff3952b2d97 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:45.635334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 486/967] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Aug 4 07:24:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:45.636168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a5bc8-7d08-40ba-9e48-4ff3952b2d97 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:45.656364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e355d70-87a6-446a-94ab-f6e40cf13205 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:45.709415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5046425-18a4-4810-94e3-337775bd2b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:45.711186 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5046425-18a4-4810-94e3-337775bd2b9f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:45.711974 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5046425-18a4-4810-94e3-337775bd2b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:45.712493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5046425-18a4-4810-94e3-337775bd2b9f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:45.713372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5046425-18a4-4810-94e3-337775bd2b9f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:45.715283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5046425-18a4-4810-94e3-337775bd2b9f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d5046425-18a4-4810-94e3-337775bd2b9f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:45.726616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:45.727755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:45.728670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:45.733286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 482/968] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Aug 4 07:24:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:45.734398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5046425-18a4-4810-94e3-337775bd2b9f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:45.746350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:45.766361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a5bc8-7d08-40ba-9e48-4ff3952b2d97 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:45.812948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:45.820101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8a1f4-f05b-415d-893c-03fda494549e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:45.820690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a1f4-f05b-415d-893c-03fda494549e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:45.820968 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a1f4-f05b-415d-893c-03fda494549e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:45.821079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a1f4-f05b-415d-893c-03fda494549e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:45.822004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8a1f4-f05b-415d-893c-03fda494549e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:45.823095 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8a1f4-f05b-415d-893c-03fda494549e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:45.824548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8a1f4-f05b-415d-893c-03fda494549e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:45.836190 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 487/969] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Aug 4 07:24:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:45.842628 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8a1f4-f05b-415d-893c-03fda494549e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:45.860314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5046425-18a4-4810-94e3-337775bd2b9f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:45.900193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0261e3a-dfd4-49fc-9465-c93eb8217be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:45.901029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0261e3a-dfd4-49fc-9465-c93eb8217be6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:45.901314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0261e3a-dfd4-49fc-9465-c93eb8217be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:45.901591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0261e3a-dfd4-49fc-9465-c93eb8217be6 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:45.902281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0261e3a-dfd4-49fc-9465-c93eb8217be6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:45.902589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0261e3a-dfd4-49fc-9465-c93eb8217be6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:45.904500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0261e3a-dfd4-49fc-9465-c93eb8217be6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:45.914110 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 483/970] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Aug 4 07:24:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:45.920551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0261e3a-dfd4-49fc-9465-c93eb8217be6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:45.967667 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8a1f4-f05b-415d-893c-03fda494549e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:46.007324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:46.007755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81508abe-95c0-4e5c-b619-61dd824e0cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:46.008641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508abe-95c0-4e5c-b619-61dd824e0cbd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.008770 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508abe-95c0-4e5c-b619-61dd824e0cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.008960 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508abe-95c0-4e5c-b619-61dd824e0cbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.020757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81508abe-95c0-4e5c-b619-61dd824e0cbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:46.027434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81508abe-95c0-4e5c-b619-61dd824e0cbd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:46.037590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0261e3a-dfd4-49fc-9465-c93eb8217be6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-81508abe-95c0-4e5c-b619-61dd824e0cbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:46.037802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:46.038875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:46.039836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:46.039836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:46.039836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:46.039836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:46.039836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 488/971] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:46.040176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57e589-eb16-419f-9de3-e615e7925df8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.040261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57e589-eb16-419f-9de3-e615e7925df8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.040464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57e589-eb16-419f-9de3-e615e7925df8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.041692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 489/972] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:46.044273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57e589-eb16-419f-9de3-e615e7925df8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:46.078084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1d9e-0885-4680-9cf6-7428c02617ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:46.078705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1d9e-0885-4680-9cf6-7428c02617ec None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.079018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1d9e-0885-4680-9cf6-7428c02617ec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.079312 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1d9e-0885-4680-9cf6-7428c02617ec None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.080028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a1d9e-0885-4680-9cf6-7428c02617ec None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:46.080328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a1d9e-0885-4680-9cf6-7428c02617ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:46.081686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a1d9e-0885-4680-9cf6-7428c02617ec None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:46.086404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 484/973] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Aug 4 07:24:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:46.087615 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cf64c-ded8-4e40-ae4c-02b9fda2ec88 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.087910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cf64c-ded8-4e40-ae4c-02b9fda2ec88 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.088287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cf64c-ded8-4e40-ae4c-02b9fda2ec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.095215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:46.106194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-025cf64c-ded8-4e40-ae4c-02b9fda2ec88 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:46.107648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-025cf64c-ded8-4e40-ae4c-02b9fda2ec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:46.149114 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57e589-eb16-419f-9de3-e615e7925df8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:46.174151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 485/974] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:46.177001 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cf64c-ded8-4e40-ae4c-02b9fda2ec88 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:46.183614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69cbb0-c128-4456-8a09-020c55749f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:46.184266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69cbb0-c128-4456-8a09-020c55749f0a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.184553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69cbb0-c128-4456-8a09-020c55749f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.184828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69cbb0-c128-4456-8a09-020c55749f0a None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.185516 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69cbb0-c128-4456-8a09-020c55749f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:46.186111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69cbb0-c128-4456-8a09-020c55749f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:46.187183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69cbb0-c128-4456-8a09-020c55749f0a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:46.199674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 490/975] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:46.200569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69cbb0-c128-4456-8a09-020c55749f0a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:46.249060 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:46.282172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cf64c-ded8-4e40-ae4c-02b9fda2ec88 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:46.299727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69cbb0-c128-4456-8a09-020c55749f0a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:46.306612 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:46.317143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff7e76-9535-41ce-8517-357264dfe228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:46.317763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7e76-9535-41ce-8517-357264dfe228 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.318081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7e76-9535-41ce-8517-357264dfe228 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.318356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7e76-9535-41ce-8517-357264dfe228 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.319071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff7e76-9535-41ce-8517-357264dfe228 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:46.320544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff7e76-9535-41ce-8517-357264dfe228 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:46.333866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99bd26-bcfe-4f54-9076-e365abd70e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:46.334580 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99bd26-bcfe-4f54-9076-e365abd70e33 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.334864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99bd26-bcfe-4f54-9076-e365abd70e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.335139 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99bd26-bcfe-4f54-9076-e365abd70e33 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.335763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99bd26-bcfe-4f54-9076-e365abd70e33 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:46.337186 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99bd26-bcfe-4f54-9076-e365abd70e33 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fc99bd26-bcfe-4f54-9076-e365abd70e33 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:46.346090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:46.347488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:46.348593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 491/976] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:46.349593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99bd26-bcfe-4f54-9076-e365abd70e33 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:46.429369 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 486/977] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:24:46 2022] POST /identity/v3/endpoints => generated 349 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:46.430455 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff7e76-9535-41ce-8517-357264dfe228 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:46.448575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99bd26-bcfe-4f54-9076-e365abd70e33 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:46.496475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01592dca-b0f2-462d-8d08-ddf06ee90efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:46.497248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01592dca-b0f2-462d-8d08-ddf06ee90efd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.497529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01592dca-b0f2-462d-8d08-ddf06ee90efd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.497936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01592dca-b0f2-462d-8d08-ddf06ee90efd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.507379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01592dca-b0f2-462d-8d08-ddf06ee90efd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:46.508892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01592dca-b0f2-462d-8d08-ddf06ee90efd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-01592dca-b0f2-462d-8d08-ddf06ee90efd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:46.522099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:46.523149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:46.524236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:46.524236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:46.524236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:46.524236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:46.524236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:46.524236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:46.524236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:46.524236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:46.524236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 07:24:46.524236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:46.524236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 492/978] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Aug 4 07:24:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:46.529462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01592dca-b0f2-462d-8d08-ddf06ee90efd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:46.543592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff7e76-9535-41ce-8517-357264dfe228 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:46.561848 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:46.570296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:46.583782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353dd50-3402-4586-96da-3baaaf1e5118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:46.584438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353dd50-3402-4586-96da-3baaaf1e5118 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.584721 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353dd50-3402-4586-96da-3baaaf1e5118 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.584998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353dd50-3402-4586-96da-3baaaf1e5118 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.585689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353dd50-3402-4586-96da-3baaaf1e5118 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:46.585992 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353dd50-3402-4586-96da-3baaaf1e5118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:46.587405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353dd50-3402-4586-96da-3baaaf1e5118 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:46.592654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 487/979] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:46.598979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353dd50-3402-4586-96da-3baaaf1e5118 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:46.650522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01592dca-b0f2-462d-8d08-ddf06ee90efd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:46.686985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92beff8d-ab7b-45e5-9c0a-c84bb2dc2e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:46.687575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92beff8d-ab7b-45e5-9c0a-c84bb2dc2e57 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.687862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92beff8d-ab7b-45e5-9c0a-c84bb2dc2e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.688197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92beff8d-ab7b-45e5-9c0a-c84bb2dc2e57 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.689306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92beff8d-ab7b-45e5-9c0a-c84bb2dc2e57 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:46.689641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92beff8d-ab7b-45e5-9c0a-c84bb2dc2e57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:46.691081 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92beff8d-ab7b-45e5-9c0a-c84bb2dc2e57 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:46.709295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 493/980] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Aug 4 07:24:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:46.710601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353dd50-3402-4586-96da-3baaaf1e5118 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:46.714838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92beff8d-ab7b-45e5-9c0a-c84bb2dc2e57 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:46.750387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b4358-6a9b-46fc-96ee-fe2aa4c29d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:46.751574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b4358-6a9b-46fc-96ee-fe2aa4c29d71 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.751857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b4358-6a9b-46fc-96ee-fe2aa4c29d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.752161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b4358-6a9b-46fc-96ee-fe2aa4c29d71 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.752839 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b4358-6a9b-46fc-96ee-fe2aa4c29d71 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:46.753142 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b4358-6a9b-46fc-96ee-fe2aa4c29d71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:46.754497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b4358-6a9b-46fc-96ee-fe2aa4c29d71 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:46.764510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 488/981] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:46.765463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b4358-6a9b-46fc-96ee-fe2aa4c29d71 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:46.804135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:46.820707 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92beff8d-ab7b-45e5-9c0a-c84bb2dc2e57 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:46.821902 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:46.858381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d47b8c-01ef-4e8e-bebe-a555db8708bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:46.858997 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47b8c-01ef-4e8e-bebe-a555db8708bf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.859278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47b8c-01ef-4e8e-bebe-a555db8708bf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.859557 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47b8c-01ef-4e8e-bebe-a555db8708bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.876660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b4358-6a9b-46fc-96ee-fe2aa4c29d71 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:46.912288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5990c-d08f-41b4-a94f-0e9e6d022060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:46.912914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5990c-d08f-41b4-a94f-0e9e6d022060 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:46.914820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5990c-d08f-41b4-a94f-0e9e6d022060 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:46.914820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5990c-d08f-41b4-a94f-0e9e6d022060 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:46.914820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5990c-d08f-41b4-a94f-0e9e6d022060 None admin] RBAC: Adding query filter params (interface=public, service_id=3592e75a10514f848b03767ed3f336a4, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:46.914820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5990c-d08f-41b4-a94f-0e9e6d022060 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:46.915895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5990c-d08f-41b4-a94f-0e9e6d022060 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:46.921478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 489/982] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Aug 4 07:24:46 2022] GET /identity/v3/endpoints?service_id=3592e75a10514f848b03767ed3f336a4&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:46.927649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5990c-d08f-41b4-a94f-0e9e6d022060 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:46.936298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d47b8c-01ef-4e8e-bebe-a555db8708bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:46.937819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d47b8c-01ef-4e8e-bebe-a555db8708bf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:47.034434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5990c-d08f-41b4-a94f-0e9e6d022060 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:47.038731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:47.048193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:47.056326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 494/983] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:46 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:24:47.062239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d47b8c-01ef-4e8e-bebe-a555db8708bf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:47.072915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8c8db-ef71-48f8-94a9-ba25e8e3ea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:47.073581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8c8db-ef71-48f8-94a9-ba25e8e3ea0d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:47.073867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8c8db-ef71-48f8-94a9-ba25e8e3ea0d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:47.074139 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8c8db-ef71-48f8-94a9-ba25e8e3ea0d None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:47.074801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8c8db-ef71-48f8-94a9-ba25e8e3ea0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:47.076191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8c8db-ef71-48f8-94a9-ba25e8e3ea0d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:47.083467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 490/984] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Aug 4 07:24:46 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:24:47.178061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d47b8c-01ef-4e8e-bebe-a555db8708bf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:47.220450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6a363-9c83-4d90-831d-997714ea6677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:47.220450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6a363-9c83-4d90-831d-997714ea6677 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:47.220450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6a363-9c83-4d90-831d-997714ea6677 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:47.220450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6a363-9c83-4d90-831d-997714ea6677 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:47.227523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6a363-9c83-4d90-831d-997714ea6677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:47.229006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6a363-9c83-4d90-831d-997714ea6677 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0ad6a363-9c83-4d90-831d-997714ea6677 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:47.244932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:47.245893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:47.246941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:47.246941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:47.246941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:47.246941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:47.246941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 495/985] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:47.252247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8c8db-ef71-48f8-94a9-ba25e8e3ea0d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:47.351368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8c8db-ef71-48f8-94a9-ba25e8e3ea0d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:47.384232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073250a-ae7c-48c2-ab6f-81dbba54e3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:47.384848 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073250a-ae7c-48c2-ab6f-81dbba54e3ac None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:47.385127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073250a-ae7c-48c2-ab6f-81dbba54e3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:47.385400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073250a-ae7c-48c2-ab6f-81dbba54e3ac None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:47.386081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073250a-ae7c-48c2-ab6f-81dbba54e3ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:47.386384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073250a-ae7c-48c2-ab6f-81dbba54e3ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:47.387736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073250a-ae7c-48c2-ab6f-81dbba54e3ac None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:47.399838 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 491/986] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:47.405170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6a363-9c83-4d90-831d-997714ea6677 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:47.440938 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:47.506669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6a363-9c83-4d90-831d-997714ea6677 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:47.543670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680d81c-93e8-4bbf-be15-ab33f8a73bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:47.544346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680d81c-93e8-4bbf-be15-ab33f8a73bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:47.544616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680d81c-93e8-4bbf-be15-ab33f8a73bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:47.544889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680d81c-93e8-4bbf-be15-ab33f8a73bd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:47.551835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680d81c-93e8-4bbf-be15-ab33f8a73bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:47.553304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680d81c-93e8-4bbf-be15-ab33f8a73bd4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4680d81c-93e8-4bbf-be15-ab33f8a73bd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:47.565518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:47.566458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:47.568211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:47.568211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:47.568211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:47.568211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:47.568211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:47.568211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:47.568211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:47.568211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:47.568211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 496/987] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:47.572669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073250a-ae7c-48c2-ab6f-81dbba54e3ac None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:47.617993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:47.679020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073250a-ae7c-48c2-ab6f-81dbba54e3ac None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:47.712904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c455e7-0af1-46d8-b3b7-6763d378712b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:47.713651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c455e7-0af1-46d8-b3b7-6763d378712b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:47.714041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c455e7-0af1-46d8-b3b7-6763d378712b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:47.714419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c455e7-0af1-46d8-b3b7-6763d378712b None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:47.715272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c455e7-0af1-46d8-b3b7-6763d378712b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:47.715684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c455e7-0af1-46d8-b3b7-6763d378712b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:47.717288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c455e7-0af1-46d8-b3b7-6763d378712b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:47.734756 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 492/988] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:47.736604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680d81c-93e8-4bbf-be15-ab33f8a73bd4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:47.833527 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:47.840145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680d81c-93e8-4bbf-be15-ab33f8a73bd4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:47.874355 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a92634-0443-4cf4-83fe-8ce18b7a00c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:47.874991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a92634-0443-4cf4-83fe-8ce18b7a00c1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:47.875276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a92634-0443-4cf4-83fe-8ce18b7a00c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:47.875552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a92634-0443-4cf4-83fe-8ce18b7a00c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:47.877012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a92634-0443-4cf4-83fe-8ce18b7a00c1 None admin] RBAC: Adding query filter params (role.id=eefaec6a74bd4d5c93eca0ae44a177fa, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=0c30c2fd1c4648308fac6606526451ec) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:47.877322 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a92634-0443-4cf4-83fe-8ce18b7a00c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:47.878745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a92634-0443-4cf4-83fe-8ce18b7a00c1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:47.892246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 497/989] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:47 2022] GET /identity/v3/role_assignments?user.id=0c30c2fd1c4648308fac6606526451ec&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=eefaec6a74bd4d5c93eca0ae44a177fa => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:48.495707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727a89c-8ceb-4354-bc61-a45faaa3c323 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:48.496273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727a89c-8ceb-4354-bc61-a45faaa3c323 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:48.496273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727a89c-8ceb-4354-bc61-a45faaa3c323 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:48.497807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 493/990] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:48.503284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16a84a-9a21-4283-9c38-48bc1a1d994e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:48.503575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16a84a-9a21-4283-9c38-48bc1a1d994e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:48.503855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16a84a-9a21-4283-9c38-48bc1a1d994e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:48.521600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8b16a84a-9a21-4283-9c38-48bc1a1d994e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:48.522775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8b16a84a-9a21-4283-9c38-48bc1a1d994e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:48.593834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 498/991] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:48.616909 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727a89c-8ceb-4354-bc61-a45faaa3c323 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:48.714856 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:48.720267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727a89c-8ceb-4354-bc61-a45faaa3c323 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:48.755720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76690347-8520-4953-ae72-3c65f2954d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:48.756309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76690347-8520-4953-ae72-3c65f2954d56 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:48.756591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76690347-8520-4953-ae72-3c65f2954d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:48.756867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76690347-8520-4953-ae72-3c65f2954d56 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:48.757495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76690347-8520-4953-ae72-3c65f2954d56 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:48.758860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76690347-8520-4953-ae72-3c65f2954d56 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-76690347-8520-4953-ae72-3c65f2954d56 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:48.767644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:48.768835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:48.769684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:48.772357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 494/992] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Aug 4 07:24:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:48.774735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16a84a-9a21-4283-9c38-48bc1a1d994e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:48.865316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:48.875727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16a84a-9a21-4283-9c38-48bc1a1d994e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:48.910111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daad82-3624-4fc9-9958-9801767ae432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:48.910729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daad82-3624-4fc9-9958-9801767ae432 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:48.911011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daad82-3624-4fc9-9958-9801767ae432 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:48.911287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daad82-3624-4fc9-9958-9801767ae432 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:48.911959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daad82-3624-4fc9-9958-9801767ae432 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:48.912296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daad82-3624-4fc9-9958-9801767ae432 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:48.913700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daad82-3624-4fc9-9958-9801767ae432 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:48.918758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 499/993] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Aug 4 07:24:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:48.924885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76690347-8520-4953-ae72-3c65f2954d56 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:49.035700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76690347-8520-4953-ae72-3c65f2954d56 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:49.069245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429513e-f50a-41cc-8b9f-a5356df3446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:49.069867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429513e-f50a-41cc-8b9f-a5356df3446c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:49.070150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429513e-f50a-41cc-8b9f-a5356df3446c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:49.070425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429513e-f50a-41cc-8b9f-a5356df3446c None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:49.071102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429513e-f50a-41cc-8b9f-a5356df3446c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:49.071452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429513e-f50a-41cc-8b9f-a5356df3446c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:49.072849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429513e-f50a-41cc-8b9f-a5356df3446c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:49.077836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 495/994] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Aug 4 07:24:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:49.083506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daad82-3624-4fc9-9958-9801767ae432 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:49.136081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d504b-c3fd-483d-bd3e-8b18be296e00 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:49.136426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d504b-c3fd-483d-bd3e-8b18be296e00 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:49.136649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d504b-c3fd-483d-bd3e-8b18be296e00 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:49.138248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 496/995] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:49.143687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecca116-5239-4cb6-846c-f2759c33d3ed None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:49.143972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecca116-5239-4cb6-846c-f2759c33d3ed None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:49.144273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecca116-5239-4cb6-846c-f2759c33d3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:49.164339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2ecca116-5239-4cb6-846c-f2759c33d3ed None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:49.165530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2ecca116-5239-4cb6-846c-f2759c33d3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:49.187086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:49.192074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daad82-3624-4fc9-9958-9801767ae432 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:49.229060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bbfad-66ec-4333-95a9-8ab23c14a1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:49.229976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bbfad-66ec-4333-95a9-8ab23c14a1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:49.230267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bbfad-66ec-4333-95a9-8ab23c14a1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:49.230540 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bbfad-66ec-4333-95a9-8ab23c14a1b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:49.231552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bbfad-66ec-4333-95a9-8ab23c14a1b0 None admin] RBAC: Adding query filter params (interface=admin, service_id=1a4ae4aa330d417fb375169e8a00a17b, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:49.231862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bbfad-66ec-4333-95a9-8ab23c14a1b0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:49.233267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bbfad-66ec-4333-95a9-8ab23c14a1b0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:49.238230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 497/996] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:49.240032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 500/997] 10.222.0.64 () {56 vars in 1249 bytes} [Thu Aug 4 07:24:49 2022] GET /identity/v3/endpoints?service_id=1a4ae4aa330d417fb375169e8a00a17b&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:49.244125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecca116-5239-4cb6-846c-f2759c33d3ed None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:49.262827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bbfad-66ec-4333-95a9-8ab23c14a1b0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:49.285375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:49.349869 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecca116-5239-4cb6-846c-f2759c33d3ed None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:49.366632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bbfad-66ec-4333-95a9-8ab23c14a1b0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:49.383450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f08bc0-cd7e-4562-bbb6-9092bf92df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:49.384105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f08bc0-cd7e-4562-bbb6-9092bf92df25 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:49.384387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f08bc0-cd7e-4562-bbb6-9092bf92df25 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:49.384660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f08bc0-cd7e-4562-bbb6-9092bf92df25 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:49.385399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f08bc0-cd7e-4562-bbb6-9092bf92df25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:49.386760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f08bc0-cd7e-4562-bbb6-9092bf92df25 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:49.394021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 498/998] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Aug 4 07:24:49 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:24:49.408623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f27b2f-cf1d-4799-9d01-6f4eacb1c65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:49.409719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f27b2f-cf1d-4799-9d01-6f4eacb1c65a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:49.410232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f27b2f-cf1d-4799-9d01-6f4eacb1c65a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:49.410718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f27b2f-cf1d-4799-9d01-6f4eacb1c65a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:49.411598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f27b2f-cf1d-4799-9d01-6f4eacb1c65a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:49.413343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f27b2f-cf1d-4799-9d01-6f4eacb1c65a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a0f27b2f-cf1d-4799-9d01-6f4eacb1c65a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:49.423820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:49.424971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 07:24:49.425885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:49.428156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 501/999] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:49.431945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f08bc0-cd7e-4562-bbb6-9092bf92df25 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:49.457559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:49.542158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f08bc0-cd7e-4562-bbb6-9092bf92df25 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:49.579576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe54165-6423-4727-ab3f-2c3d9c2f6d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:49.580395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe54165-6423-4727-ab3f-2c3d9c2f6d4f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:49.580683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe54165-6423-4727-ab3f-2c3d9c2f6d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:49.580957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe54165-6423-4727-ab3f-2c3d9c2f6d4f None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:49.581642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe54165-6423-4727-ab3f-2c3d9c2f6d4f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:49.581947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe54165-6423-4727-ab3f-2c3d9c2f6d4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:49.583315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe54165-6423-4727-ab3f-2c3d9c2f6d4f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:49.588604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 499/1000] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:49.594546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f27b2f-cf1d-4799-9d01-6f4eacb1c65a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:49.678551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:49.697236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f27b2f-cf1d-4799-9d01-6f4eacb1c65a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:49.732037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ca5cf-6632-4912-8119-b26bf9c3ad94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:49.732658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ca5cf-6632-4912-8119-b26bf9c3ad94 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:49.732940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ca5cf-6632-4912-8119-b26bf9c3ad94 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:49.733216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ca5cf-6632-4912-8119-b26bf9c3ad94 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:49.733895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ca5cf-6632-4912-8119-b26bf9c3ad94 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:49.734198 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ca5cf-6632-4912-8119-b26bf9c3ad94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:49.735674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ca5cf-6632-4912-8119-b26bf9c3ad94 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:49.740713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 502/1001] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Aug 4 07:24:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:49.746176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe54165-6423-4727-ab3f-2c3d9c2f6d4f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:49.831968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:49.848328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe54165-6423-4727-ab3f-2c3d9c2f6d4f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:49.884073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34237a-d45d-4aaf-8779-0e4dd77fe2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:49.885066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34237a-d45d-4aaf-8779-0e4dd77fe2fa None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:49.885556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34237a-d45d-4aaf-8779-0e4dd77fe2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:49.886027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34237a-d45d-4aaf-8779-0e4dd77fe2fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:49.886948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34237a-d45d-4aaf-8779-0e4dd77fe2fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:49.888554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34237a-d45d-4aaf-8779-0e4dd77fe2fa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:49.920522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e922414-2ba7-4a81-a656-dd8a1a94cb4c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:49.921073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e922414-2ba7-4a81-a656-dd8a1a94cb4c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:49.921073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e922414-2ba7-4a81-a656-dd8a1a94cb4c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:49.922476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 503/1002] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:24:49.928264 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cb11e-3d5c-4260-82fc-5a89721a1569 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:49.928607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cb11e-3d5c-4260-82fc-5a89721a1569 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:49.928835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cb11e-3d5c-4260-82fc-5a89721a1569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:49.946983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-224cb11e-3d5c-4260-82fc-5a89721a1569 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:49.948207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-224cb11e-3d5c-4260-82fc-5a89721a1569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:50.004017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 500/1003] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:24:49 2022] POST /identity/v3/endpoints => generated 334 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:50.038837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 504/1004] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:49 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:50.060122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34237a-d45d-4aaf-8779-0e4dd77fe2fa None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:50.173612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34237a-d45d-4aaf-8779-0e4dd77fe2fa None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:50.210444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5972cab-b3cf-4757-bbe1-177464245cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:50.210444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5972cab-b3cf-4757-bbe1-177464245cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:50.210444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5972cab-b3cf-4757-bbe1-177464245cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:50.210444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5972cab-b3cf-4757-bbe1-177464245cd7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:50.210829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5972cab-b3cf-4757-bbe1-177464245cd7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:50.211950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5972cab-b3cf-4757-bbe1-177464245cd7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b5972cab-b3cf-4757-bbe1-177464245cd7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:50.221363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:50.222402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:50.223461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 501/1005] 10.222.0.64 () {56 vars in 1105 bytes} [Thu Aug 4 07:24:50 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:50.228975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cb11e-3d5c-4260-82fc-5a89721a1569 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:50.304774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:50.333814 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cb11e-3d5c-4260-82fc-5a89721a1569 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:50.368455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07829b3a-a314-4413-b3e9-4eeb1da7f80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:50.369096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829b3a-a314-4413-b3e9-4eeb1da7f80a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:50.369381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829b3a-a314-4413-b3e9-4eeb1da7f80a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:50.369657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829b3a-a314-4413-b3e9-4eeb1da7f80a None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:50.370334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07829b3a-a314-4413-b3e9-4eeb1da7f80a None admin] RBAC: Adding query filter params (name=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:50.370642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07829b3a-a314-4413-b3e9-4eeb1da7f80a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:50.372041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07829b3a-a314-4413-b3e9-4eeb1da7f80a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:50.377251 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 505/1006] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:50.382675 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5972cab-b3cf-4757-bbe1-177464245cd7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:50.387102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:50.485854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5972cab-b3cf-4757-bbe1-177464245cd7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:50.520910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c11831-659c-4da5-9799-1c2d2d93553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:50.521552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c11831-659c-4da5-9799-1c2d2d93553f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:50.521833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c11831-659c-4da5-9799-1c2d2d93553f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:50.522106 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c11831-659c-4da5-9799-1c2d2d93553f None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:50.522806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c11831-659c-4da5-9799-1c2d2d93553f None admin] RBAC: Adding query filter params (type=image) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:50.523109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c11831-659c-4da5-9799-1c2d2d93553f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:50.524548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c11831-659c-4da5-9799-1c2d2d93553f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:50.536020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 502/1007] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:51.434915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed11ca-9229-4fdf-b61e-cf22d0bf6818 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:51.435418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed11ca-9229-4fdf-b61e-cf22d0bf6818 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:51.435515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed11ca-9229-4fdf-b61e-cf22d0bf6818 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:51.437106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 506/1008] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:51.442620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb599c7-6aa1-4230-a3ab-474a8da8e854 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:51.443010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb599c7-6aa1-4230-a3ab-474a8da8e854 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:51.443174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb599c7-6aa1-4230-a3ab-474a8da8e854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:51.461468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bbb599c7-6aa1-4230-a3ab-474a8da8e854 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:51.462646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bbb599c7-6aa1-4230-a3ab-474a8da8e854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:51.524355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:51.530482 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 503/1009] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:51 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:51.553998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed11ca-9229-4fdf-b61e-cf22d0bf6818 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:51.659213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed11ca-9229-4fdf-b61e-cf22d0bf6818 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:51.694783 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea5b69-285f-4738-a163-1edc2cc82a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:51.695343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea5b69-285f-4738-a163-1edc2cc82a03 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:51.695717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea5b69-285f-4738-a163-1edc2cc82a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:51.696011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea5b69-285f-4738-a163-1edc2cc82a03 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:51.696645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea5b69-285f-4738-a163-1edc2cc82a03 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:51.698082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea5b69-285f-4738-a163-1edc2cc82a03 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:51.700718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e4ea5b69-285f-4738-a163-1edc2cc82a03 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:51.708471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:51.709452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 07:24:51.710328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:51.711727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 507/1010] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Aug 4 07:24:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:51.715780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb599c7-6aa1-4230-a3ab-474a8da8e854 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:51.818454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb599c7-6aa1-4230-a3ab-474a8da8e854 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:51.852546 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb0eb9-0161-42eb-9881-81aa5b40afd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:51.853660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb0eb9-0161-42eb-9881-81aa5b40afd2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:51.853952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb0eb9-0161-42eb-9881-81aa5b40afd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:51.854230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb0eb9-0161-42eb-9881-81aa5b40afd2 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:51.854910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb0eb9-0161-42eb-9881-81aa5b40afd2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:51.855217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb0eb9-0161-42eb-9881-81aa5b40afd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:51.856934 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb0eb9-0161-42eb-9881-81aa5b40afd2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:51.866907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 504/1011] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Aug 4 07:24:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:51.868067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea5b69-285f-4738-a163-1edc2cc82a03 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:51.978883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea5b69-285f-4738-a163-1edc2cc82a03 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:52.012541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ae1ca-6fc9-4824-9ba9-457cd0670831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:52.013158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ae1ca-6fc9-4824-9ba9-457cd0670831 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:52.013440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ae1ca-6fc9-4824-9ba9-457cd0670831 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:52.013713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ae1ca-6fc9-4824-9ba9-457cd0670831 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:52.014385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ae1ca-6fc9-4824-9ba9-457cd0670831 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:52.014687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ae1ca-6fc9-4824-9ba9-457cd0670831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:52.016123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ae1ca-6fc9-4824-9ba9-457cd0670831 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:52.026024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 508/1012] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Aug 4 07:24:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:52.027239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb0eb9-0161-42eb-9881-81aa5b40afd2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:52.091255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:52.130161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb0eb9-0161-42eb-9881-81aa5b40afd2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:52.152547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63589aa-f6dd-4e29-9adb-4029ae2766a2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:52.152863 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63589aa-f6dd-4e29-9adb-4029ae2766a2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:52.153130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63589aa-f6dd-4e29-9adb-4029ae2766a2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:52.154601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 509/1013] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:24:52.159820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d4cfd-35c4-4679-99f5-3921de066d7b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:52.160132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d4cfd-35c4-4679-99f5-3921de066d7b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:52.160416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d4cfd-35c4-4679-99f5-3921de066d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:52.167004 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34559bb-26f1-48a4-b765-2a9e485392bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:52.168562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34559bb-26f1-48a4-b765-2a9e485392bf None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:52.168852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34559bb-26f1-48a4-b765-2a9e485392bf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:52.169137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34559bb-26f1-48a4-b765-2a9e485392bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:52.169871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34559bb-26f1-48a4-b765-2a9e485392bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:52.171245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34559bb-26f1-48a4-b765-2a9e485392bf None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:52.179439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9a6d4cfd-35c4-4679-99f5-3921de066d7b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:52.180660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9a6d4cfd-35c4-4679-99f5-3921de066d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:52.240444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 505/1014] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:24:52 2022] POST /identity/v3/endpoints => generated 325 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:52.250221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:52.267308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 510/1015] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:52 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:52.292343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34559bb-26f1-48a4-b765-2a9e485392bf None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:52.401612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34559bb-26f1-48a4-b765-2a9e485392bf None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:52.426805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:52.439199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd83f-21c4-4724-a5dd-5d1c9a2a8c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:52.439852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd83f-21c4-4724-a5dd-5d1c9a2a8c76 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:52.440175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd83f-21c4-4724-a5dd-5d1c9a2a8c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:52.440456 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd83f-21c4-4724-a5dd-5d1c9a2a8c76 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:52.451073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcd83f-21c4-4724-a5dd-5d1c9a2a8c76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ecbcd83f-21c4-4724-a5dd-5d1c9a2a8c76 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:52.456621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:52.457668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:52.458518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 506/1016] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Aug 4 07:24:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:52.465207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d4cfd-35c4-4679-99f5-3921de066d7b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:52.525890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdde205-7144-4c13-bdb7-6c96a5093b24 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:52.526065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdde205-7144-4c13-bdb7-6c96a5093b24 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:52.526345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdde205-7144-4c13-bdb7-6c96a5093b24 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:52.527807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 507/1017] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:24:52.533285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8667240-5a32-48d5-a2ab-f207de5b49a5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:52.533576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8667240-5a32-48d5-a2ab-f207de5b49a5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:52.533857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8667240-5a32-48d5-a2ab-f207de5b49a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:52.553430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c8667240-5a32-48d5-a2ab-f207de5b49a5 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:52.554619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c8667240-5a32-48d5-a2ab-f207de5b49a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:52.571613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d4cfd-35c4-4679-99f5-3921de066d7b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:52.609113 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e6567-b24f-49b0-819b-c5a3f5aa3553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:52.609735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6567-b24f-49b0-819b-c5a3f5aa3553 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:52.610020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6567-b24f-49b0-819b-c5a3f5aa3553 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:52.610298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6567-b24f-49b0-819b-c5a3f5aa3553 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:52.611018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e6567-b24f-49b0-819b-c5a3f5aa3553 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:52.611323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e6567-b24f-49b0-819b-c5a3f5aa3553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:52.612854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e6567-b24f-49b0-819b-c5a3f5aa3553 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:52.625054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 508/1018] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:52 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:52.627140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 511/1019] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Aug 4 07:24:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:52.628928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8667240-5a32-48d5-a2ab-f207de5b49a5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:52.648427 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e6567-b24f-49b0-819b-c5a3f5aa3553 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:52.705555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:52.736261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8667240-5a32-48d5-a2ab-f207de5b49a5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:52.750738 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:52.759209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e6567-b24f-49b0-819b-c5a3f5aa3553 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:52.774148 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f79235-c31c-4239-9ead-4603668662bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:52.774793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f79235-c31c-4239-9ead-4603668662bc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:52.775079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f79235-c31c-4239-9ead-4603668662bc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:52.775355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f79235-c31c-4239-9ead-4603668662bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:52.787132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f79235-c31c-4239-9ead-4603668662bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:52.788549 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f79235-c31c-4239-9ead-4603668662bc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:52.794220 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8abc79-2b93-440b-89dc-05d85c812e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:52.794840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8abc79-2b93-440b-89dc-05d85c812e3d None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:52.795126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8abc79-2b93-440b-89dc-05d85c812e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:52.795410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8abc79-2b93-440b-89dc-05d85c812e3d None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:52.796135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8abc79-2b93-440b-89dc-05d85c812e3d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:52.798041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8abc79-2b93-440b-89dc-05d85c812e3d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f5f79235-c31c-4239-9ead-4603668662bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:52.804682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:52.805760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:52.806679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:52.806679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:52.806679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:52.806679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:52.806679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 509/1020] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:52.811339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f79235-c31c-4239-9ead-4603668662bc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:52.878708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 512/1021] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:24:52 2022] POST /identity/v3/services => generated 242 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:52.920877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f79235-c31c-4239-9ead-4603668662bc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:52.976849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cc11c-e450-407f-90bf-f90665e107a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:52.977481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cc11c-e450-407f-90bf-f90665e107a3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:52.977765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cc11c-e450-407f-90bf-f90665e107a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:52.978041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cc11c-e450-407f-90bf-f90665e107a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:52.978781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cc11c-e450-407f-90bf-f90665e107a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:52.979077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cc11c-e450-407f-90bf-f90665e107a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:52.980620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cc11c-e450-407f-90bf-f90665e107a3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:52.998703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 510/1022] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:53.000292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8abc79-2b93-440b-89dc-05d85c812e3d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:53.107528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8abc79-2b93-440b-89dc-05d85c812e3d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:53.121224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:53.142243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4b4df-9b36-4ff1-bb02-ef49ce2b3d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:53.142887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4b4df-9b36-4ff1-bb02-ef49ce2b3d71 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:53.143171 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4b4df-9b36-4ff1-bb02-ef49ce2b3d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:53.143448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4b4df-9b36-4ff1-bb02-ef49ce2b3d71 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:53.150831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4b4df-9b36-4ff1-bb02-ef49ce2b3d71 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:53.152372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4b4df-9b36-4ff1-bb02-ef49ce2b3d71 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-54a4b4df-9b36-4ff1-bb02-ef49ce2b3d71 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:53.163416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:53.164666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:53.165620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:53.165620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:53.165620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:53.165620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:53.165620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:53.165620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:53.165620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:53.165620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:53.165620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:53.165620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:53.165620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 513/1023] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Aug 4 07:24:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:53.173126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cc11c-e450-407f-90bf-f90665e107a3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:53.197749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:53.278046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cc11c-e450-407f-90bf-f90665e107a3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:53.313177 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf4bb1-4b0b-4818-83e7-50e091c0802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:53.313995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf4bb1-4b0b-4818-83e7-50e091c0802a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:53.314353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf4bb1-4b0b-4818-83e7-50e091c0802a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:53.314638 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf4bb1-4b0b-4818-83e7-50e091c0802a None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:53.315444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf4bb1-4b0b-4818-83e7-50e091c0802a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:53.315758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf4bb1-4b0b-4818-83e7-50e091c0802a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:53.317224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf4bb1-4b0b-4818-83e7-50e091c0802a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:53.339476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 511/1024] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Aug 4 07:24:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:53.341146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4b4df-9b36-4ff1-bb02-ef49ce2b3d71 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:53.448696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4b4df-9b36-4ff1-bb02-ef49ce2b3d71 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:53.482741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d3e37-8148-4ba8-b678-6ac03b03bf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:53.483382 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d3e37-8148-4ba8-b678-6ac03b03bf73 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:53.483621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d3e37-8148-4ba8-b678-6ac03b03bf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:53.483894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d3e37-8148-4ba8-b678-6ac03b03bf73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:53.524283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:53.550130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d3e37-8148-4ba8-b678-6ac03b03bf73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:53.551585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d3e37-8148-4ba8-b678-6ac03b03bf73 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:53.653648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 514/1025] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:24:53 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 07:24:53.654857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf4bb1-4b0b-4818-83e7-50e091c0802a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:53.734049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:53.756607 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf4bb1-4b0b-4818-83e7-50e091c0802a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:53.790647 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c743e7e-e345-4f58-8428-ad9736f85d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:53.791294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743e7e-e345-4f58-8428-ad9736f85d2e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:53.791582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743e7e-e345-4f58-8428-ad9736f85d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:53.791862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743e7e-e345-4f58-8428-ad9736f85d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:53.803486 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c743e7e-e345-4f58-8428-ad9736f85d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:53.804964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c743e7e-e345-4f58-8428-ad9736f85d2e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0c743e7e-e345-4f58-8428-ad9736f85d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:53.819937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:53.820866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:53.821774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:53.821774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:53.821774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:53.821774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:53.822579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 512/1026] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:53.826829 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d3e37-8148-4ba8-b678-6ac03b03bf73 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:53.898343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:53.928205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d3e37-8148-4ba8-b678-6ac03b03bf73 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:53.967839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f95e0e-c44b-415f-8a3f-cbc338d51115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:53.968596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f95e0e-c44b-415f-8a3f-cbc338d51115 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:53.968975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f95e0e-c44b-415f-8a3f-cbc338d51115 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:53.969403 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f95e0e-c44b-415f-8a3f-cbc338d51115 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:53.970294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f95e0e-c44b-415f-8a3f-cbc338d51115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:53.970706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f95e0e-c44b-415f-8a3f-cbc338d51115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:53.972372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f95e0e-c44b-415f-8a3f-cbc338d51115 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:53.985464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 515/1027] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:53.990994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c743e7e-e345-4f58-8428-ad9736f85d2e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:54.096618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c743e7e-e345-4f58-8428-ad9736f85d2e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:54.124652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:54.130964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52305355-8b70-4d25-88d2-c926d58241b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:54.131603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52305355-8b70-4d25-88d2-c926d58241b5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:54.131882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52305355-8b70-4d25-88d2-c926d58241b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:54.132187 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52305355-8b70-4d25-88d2-c926d58241b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:54.138619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52305355-8b70-4d25-88d2-c926d58241b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:54.140066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52305355-8b70-4d25-88d2-c926d58241b5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-52305355-8b70-4d25-88d2-c926d58241b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:54.151810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:54.152885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:54.153796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:54.153796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:54.153796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:54.153796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:54.153796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:54.153796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:54.153796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:54.153796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:54.154917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 513/1028] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:54.159955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f95e0e-c44b-415f-8a3f-cbc338d51115 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:54.260135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f95e0e-c44b-415f-8a3f-cbc338d51115 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:54.289881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:54.295208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd3017-35f8-4758-84ef-e539b130f76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:54.295818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd3017-35f8-4758-84ef-e539b130f76a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:54.296128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd3017-35f8-4758-84ef-e539b130f76a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:54.296448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd3017-35f8-4758-84ef-e539b130f76a None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:54.297244 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd3017-35f8-4758-84ef-e539b130f76a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:54.298144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd3017-35f8-4758-84ef-e539b130f76a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:54.299909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd3017-35f8-4758-84ef-e539b130f76a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:54.307351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54554f-3349-4f81-86e3-98ac271bec08 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:54.307643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54554f-3349-4f81-86e3-98ac271bec08 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:54.317535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54554f-3349-4f81-86e3-98ac271bec08 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:54.317535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 514/1029] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:54.317720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 516/1030] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:54.318172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd3017-35f8-4758-84ef-e539b130f76a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:54.318599 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d01ee-5af5-460a-bfd5-f62e2f0abab2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:54.318884 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d01ee-5af5-460a-bfd5-f62e2f0abab2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:54.319157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d01ee-5af5-460a-bfd5-f62e2f0abab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:54.338059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b91d01ee-5af5-460a-bfd5-f62e2f0abab2 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:54.339237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b91d01ee-5af5-460a-bfd5-f62e2f0abab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:54.408661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 515/1031] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:54.430010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd3017-35f8-4758-84ef-e539b130f76a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:54.431737 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d01ee-5af5-460a-bfd5-f62e2f0abab2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:54.465109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31de18-d01d-48c7-b13c-e314bfbb354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:54.465876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31de18-d01d-48c7-b13c-e314bfbb354f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:54.466161 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31de18-d01d-48c7-b13c-e314bfbb354f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:54.466821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31de18-d01d-48c7-b13c-e314bfbb354f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:54.467869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31de18-d01d-48c7-b13c-e314bfbb354f None admin] RBAC: Adding query filter params (role.id=eefaec6a74bd4d5c93eca0ae44a177fa, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=4eedd711c6114de1bfcdf692ce058794) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:54.468201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31de18-d01d-48c7-b13c-e314bfbb354f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:54.469988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31de18-d01d-48c7-b13c-e314bfbb354f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:54.479130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 517/1032] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:24:54 2022] GET /identity/v3/role_assignments?user.id=4eedd711c6114de1bfcdf692ce058794&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=eefaec6a74bd4d5c93eca0ae44a177fa => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:54.551563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d01ee-5af5-460a-bfd5-f62e2f0abab2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:54.574580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:54.590025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d36683-4164-4fbc-b485-c87659a774f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:54.590832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d36683-4164-4fbc-b485-c87659a774f3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:54.591116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d36683-4164-4fbc-b485-c87659a774f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:54.591392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d36683-4164-4fbc-b485-c87659a774f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:54.592642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d36683-4164-4fbc-b485-c87659a774f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:54.594250 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d36683-4164-4fbc-b485-c87659a774f3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:54.600776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 516/1033] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:54.602043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31de18-d01d-48c7-b13c-e314bfbb354f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:54.663789 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:54.709832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31de18-d01d-48c7-b13c-e314bfbb354f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:54.743476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d32e27-68a9-434c-bf55-1fa280cf2115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:54.744122 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d32e27-68a9-434c-bf55-1fa280cf2115 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:54.744406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d32e27-68a9-434c-bf55-1fa280cf2115 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:54.744682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d32e27-68a9-434c-bf55-1fa280cf2115 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:54.745381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d32e27-68a9-434c-bf55-1fa280cf2115 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:54.746807 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d32e27-68a9-434c-bf55-1fa280cf2115 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:54.870213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 518/1034] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:24:54 2022] POST /identity/v3/projects => generated 311 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:24:54.933613 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821d9e3-015b-4cda-afaf-d2a97bae7862 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:54.934141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821d9e3-015b-4cda-afaf-d2a97bae7862 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:54.934252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821d9e3-015b-4cda-afaf-d2a97bae7862 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:54.935769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 517/1035] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:54.942563 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0381c0-59e7-45ca-9779-885d7cdcd1af None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:54.942861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0381c0-59e7-45ca-9779-885d7cdcd1af None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:54.943147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0381c0-59e7-45ca-9779-885d7cdcd1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:54.971067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3c0381c0-59e7-45ca-9779-885d7cdcd1af None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:54.972309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3c0381c0-59e7-45ca-9779-885d7cdcd1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:55.047578 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 519/1036] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:55.070710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821d9e3-015b-4cda-afaf-d2a97bae7862 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:55.177271 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821d9e3-015b-4cda-afaf-d2a97bae7862 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:55.212149 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a741e3-0c49-42a3-b2e5-f597bb8fe55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:55.212791 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a741e3-0c49-42a3-b2e5-f597bb8fe55e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:55.213074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a741e3-0c49-42a3-b2e5-f597bb8fe55e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:55.213350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a741e3-0c49-42a3-b2e5-f597bb8fe55e None admin] PATH_INFO: `/v3/services/image` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:55.218797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a741e3-0c49-42a3-b2e5-f597bb8fe55e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:55.218797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a741e3-0c49-42a3-b2e5-f597bb8fe55e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-46a741e3-0c49-42a3-b2e5-f597bb8fe55e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:55.224854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:55.225863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:55.226717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 518/1037] 10.222.0.64 () {56 vars in 1105 bytes} [Thu Aug 4 07:24:55 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:55.232764 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0381c0-59e7-45ca-9779-885d7cdcd1af None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:55.322479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:55.381269 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0381c0-59e7-45ca-9779-885d7cdcd1af None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:55.416936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722b660-87c9-430e-84db-e074de28911f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:55.417556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722b660-87c9-430e-84db-e074de28911f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:55.417838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722b660-87c9-430e-84db-e074de28911f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:55.418113 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722b660-87c9-430e-84db-e074de28911f None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:55.418794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722b660-87c9-430e-84db-e074de28911f None admin] RBAC: Adding query filter params (name=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:55.419097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722b660-87c9-430e-84db-e074de28911f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:55.420502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722b660-87c9-430e-84db-e074de28911f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:55.430362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 520/1038] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:55.431309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a741e3-0c49-42a3-b2e5-f597bb8fe55e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:55.484561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:55.530489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a741e3-0c49-42a3-b2e5-f597bb8fe55e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:55.570810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f0227-9ad1-4bab-b397-e33b607200f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:55.571810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f0227-9ad1-4bab-b397-e33b607200f2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:55.572349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f0227-9ad1-4bab-b397-e33b607200f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:55.572773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f0227-9ad1-4bab-b397-e33b607200f2 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:55.573677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f0227-9ad1-4bab-b397-e33b607200f2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:55.574145 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f0227-9ad1-4bab-b397-e33b607200f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:55.575763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f0227-9ad1-4bab-b397-e33b607200f2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:55.582198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 519/1039] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:55.588431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722b660-87c9-430e-84db-e074de28911f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:55.689816 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722b660-87c9-430e-84db-e074de28911f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:55.691252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:55.724098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8feeb-c720-435c-9854-cdbf88b601c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:55.724714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8feeb-c720-435c-9854-cdbf88b601c8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:55.724996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8feeb-c720-435c-9854-cdbf88b601c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:55.725272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8feeb-c720-435c-9854-cdbf88b601c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:55.726019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8feeb-c720-435c-9854-cdbf88b601c8 None admin] RBAC: Adding query filter params (interface=public, service_id=f4d3131854c541478f358a7e4f669dd5, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:55.726329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8feeb-c720-435c-9854-cdbf88b601c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:55.727688 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8feeb-c720-435c-9854-cdbf88b601c8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:55.737550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 521/1040] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Aug 4 07:24:55 2022] GET /identity/v3/endpoints?service_id=f4d3131854c541478f358a7e4f669dd5&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:55.738935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f0227-9ad1-4bab-b397-e33b607200f2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:55.842686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f0227-9ad1-4bab-b397-e33b607200f2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:55.859094 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:55.877730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666bf02-ff9a-4f3b-9234-8b8946ceff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:55.878466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666bf02-ff9a-4f3b-9234-8b8946ceff12 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:55.878753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666bf02-ff9a-4f3b-9234-8b8946ceff12 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:55.879088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666bf02-ff9a-4f3b-9234-8b8946ceff12 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:55.879787 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666bf02-ff9a-4f3b-9234-8b8946ceff12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:55.881199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666bf02-ff9a-4f3b-9234-8b8946ceff12 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:55.892364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 520/1041] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Aug 4 07:24:55 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:24:56.492200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b3c5e-56db-40f5-882d-97bddd758083 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:56.492629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b3c5e-56db-40f5-882d-97bddd758083 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:56.492736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b3c5e-56db-40f5-882d-97bddd758083 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:56.494232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 522/1042] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:56.499344 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d5744-4171-43fb-917b-7f2f2f141dff None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:56.499779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d5744-4171-43fb-917b-7f2f2f141dff None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:56.499940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d5744-4171-43fb-917b-7f2f2f141dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:56.518865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5b0d5744-4171-43fb-917b-7f2f2f141dff None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:56.520062 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5b0d5744-4171-43fb-917b-7f2f2f141dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:56.586746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 521/1043] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:56.609149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b3c5e-56db-40f5-882d-97bddd758083 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:56.711720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b3c5e-56db-40f5-882d-97bddd758083 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:56.745677 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd6976-2139-4535-83e6-f6f1409be1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:56.746241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd6976-2139-4535-83e6-f6f1409be1cf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:56.746519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd6976-2139-4535-83e6-f6f1409be1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:56.746790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd6976-2139-4535-83e6-f6f1409be1cf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:56.756700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd6976-2139-4535-83e6-f6f1409be1cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2ddd6976-2139-4535-83e6-f6f1409be1cf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:56.761507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:56.762401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:56.763289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 523/1044] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Aug 4 07:24:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:56.769452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d5744-4171-43fb-917b-7f2f2f141dff None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:56.871206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d5744-4171-43fb-917b-7f2f2f141dff None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:56.882109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:56.905904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7259f-99f1-4ac5-85bb-ce0b2c5de556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:56.906654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7259f-99f1-4ac5-85bb-ce0b2c5de556 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:56.907439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7259f-99f1-4ac5-85bb-ce0b2c5de556 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:56.907720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7259f-99f1-4ac5-85bb-ce0b2c5de556 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:56.908478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7259f-99f1-4ac5-85bb-ce0b2c5de556 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:56.908787 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7259f-99f1-4ac5-85bb-ce0b2c5de556 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:56.910155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7259f-99f1-4ac5-85bb-ce0b2c5de556 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:56.924126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 522/1045] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Aug 4 07:24:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:56.925677 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd6976-2139-4535-83e6-f6f1409be1cf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:56.965267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:56.990960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85c582-b588-44ea-9745-e17113770fdc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:56.991265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85c582-b588-44ea-9745-e17113770fdc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:56.991532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85c582-b588-44ea-9745-e17113770fdc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:56.993041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 523/1046] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:56.997856 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2294de-7c8d-4741-a87f-1e86fcaad5eb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:56.998097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2294de-7c8d-4741-a87f-1e86fcaad5eb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:56.998382 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2294de-7c8d-4741-a87f-1e86fcaad5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:57.018198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-de2294de-7c8d-4741-a87f-1e86fcaad5eb None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:57.019711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-de2294de-7c8d-4741-a87f-1e86fcaad5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:57.043555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd6976-2139-4535-83e6-f6f1409be1cf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:57.078884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b544c86-f9d0-48c2-9673-0b7ac9e47289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:57.079593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b544c86-f9d0-48c2-9673-0b7ac9e47289 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:57.079900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b544c86-f9d0-48c2-9673-0b7ac9e47289 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:57.080215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b544c86-f9d0-48c2-9673-0b7ac9e47289 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:57.090378 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b544c86-f9d0-48c2-9673-0b7ac9e47289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:57.091816 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b544c86-f9d0-48c2-9673-0b7ac9e47289 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:57.098804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 524/1047] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3b544c86-f9d0-48c2-9673-0b7ac9e47289 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:57.107431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:57.108384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:57.109266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:57.109266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:57.109266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:57.109266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:57.112319 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 524/1048] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:57.114220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2294de-7c8d-4741-a87f-1e86fcaad5eb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:57.125056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b544c86-f9d0-48c2-9673-0b7ac9e47289 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:57.212416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:57.222622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2294de-7c8d-4741-a87f-1e86fcaad5eb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:57.224266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:57.233482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b544c86-f9d0-48c2-9673-0b7ac9e47289 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:57.259113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edcc61-60e9-40c2-aa8c-333a01826c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:57.259650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edcc61-60e9-40c2-aa8c-333a01826c11 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:57.259931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edcc61-60e9-40c2-aa8c-333a01826c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:57.260237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edcc61-60e9-40c2-aa8c-333a01826c11 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:57.260947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edcc61-60e9-40c2-aa8c-333a01826c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:57.261242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edcc61-60e9-40c2-aa8c-333a01826c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:57.262566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edcc61-60e9-40c2-aa8c-333a01826c11 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:57.271912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440c646-9a99-43ac-8689-930d17a4d348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:57.272571 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c646-9a99-43ac-8689-930d17a4d348 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:57.272853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c646-9a99-43ac-8689-930d17a4d348 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:57.273125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c646-9a99-43ac-8689-930d17a4d348 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:57.274245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440c646-9a99-43ac-8689-930d17a4d348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:57.280088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 525/1049] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:57.280238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440c646-9a99-43ac-8689-930d17a4d348 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:57.280238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 525/1050] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:57.280882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edcc61-60e9-40c2-aa8c-333a01826c11 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:57.282749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440c646-9a99-43ac-8689-930d17a4d348 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:57.384395 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440c646-9a99-43ac-8689-930d17a4d348 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:57.384882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edcc61-60e9-40c2-aa8c-333a01826c11 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:57.418127 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a398a-8eef-4db6-95d8-6d62fe670ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:57.418589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1ca82-5d24-4f72-b767-39cad392cdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:57.418799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a398a-8eef-4db6-95d8-6d62fe670ea5 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:57.419011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a398a-8eef-4db6-95d8-6d62fe670ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:57.419228 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ca82-5d24-4f72-b767-39cad392cdda None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:57.419324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a398a-8eef-4db6-95d8-6d62fe670ea5 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:57.419513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ca82-5d24-4f72-b767-39cad392cdda None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:57.419793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ca82-5d24-4f72-b767-39cad392cdda None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:57.419965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a398a-8eef-4db6-95d8-6d62fe670ea5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:57.421389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a398a-8eef-4db6-95d8-6d62fe670ea5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:57.427847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1ca82-5d24-4f72-b767-39cad392cdda None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:57.429276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1ca82-5d24-4f72-b767-39cad392cdda None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f2d1ca82-5d24-4f72-b767-39cad392cdda None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:24:57.441782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:57.442858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:57.443816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:57.443816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:57.443816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:24:57.443816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:24:57.443816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:24:57.443816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:24:57.443816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:24:57.443816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:24:57.443816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:24:57.443816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:57.443816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 526/1051] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Aug 4 07:24:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:57.449130 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1ca82-5d24-4f72-b767-39cad392cdda None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:57.474436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:57.520498 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 526/1052] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:24:57 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:24:57.552776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1ca82-5d24-4f72-b767-39cad392cdda None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:57.590526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d02f2-5110-4fed-9df9-153e194f0409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:57.591158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d02f2-5110-4fed-9df9-153e194f0409 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:57.591435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d02f2-5110-4fed-9df9-153e194f0409 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:57.591708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d02f2-5110-4fed-9df9-153e194f0409 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:57.592542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d02f2-5110-4fed-9df9-153e194f0409 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:57.592847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d02f2-5110-4fed-9df9-153e194f0409 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:57.594586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d02f2-5110-4fed-9df9-153e194f0409 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:57.614837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 527/1053] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Aug 4 07:24:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:57.620447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a398a-8eef-4db6-95d8-6d62fe670ea5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:57.623605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:57.729046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a398a-8eef-4db6-95d8-6d62fe670ea5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:57.766732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ee8d3-00cc-466b-91d9-c3f4a2e76da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:57.767380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ee8d3-00cc-466b-91d9-c3f4a2e76da7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:57.767662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ee8d3-00cc-466b-91d9-c3f4a2e76da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:57.767937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ee8d3-00cc-466b-91d9-c3f4a2e76da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:57.778808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ee8d3-00cc-466b-91d9-c3f4a2e76da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:57.780241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ee8d3-00cc-466b-91d9-c3f4a2e76da7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1f8ee8d3-00cc-466b-91d9-c3f4a2e76da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:24:57.794561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:57.795496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:57.800690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:24:57.800690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:24:57.800690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:24:57.800690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:57.800690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 527/1054] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:57.804080 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d02f2-5110-4fed-9df9-153e194f0409 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:57.887628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:57.906346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d02f2-5110-4fed-9df9-153e194f0409 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:57.941193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d24993-70c3-4b06-8d09-c8587a37c25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:57.941812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d24993-70c3-4b06-8d09-c8587a37c25d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:57.942091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d24993-70c3-4b06-8d09-c8587a37c25d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:57.942363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d24993-70c3-4b06-8d09-c8587a37c25d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:57.943274 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d24993-70c3-4b06-8d09-c8587a37c25d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:57.943614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d24993-70c3-4b06-8d09-c8587a37c25d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:57.945283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d24993-70c3-4b06-8d09-c8587a37c25d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:57.951895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b73252-9fc1-4c91-902a-5494c2540370 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:57.951895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b73252-9fc1-4c91-902a-5494c2540370 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:57.951895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b73252-9fc1-4c91-902a-5494c2540370 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:57.952367 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 528/1055] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:57.958067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253af47a-de26-4341-b116-9421a9832e31 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:57.958352 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253af47a-de26-4341-b116-9421a9832e31 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:57.958629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253af47a-de26-4341-b116-9421a9832e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:57.971239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 528/1056] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:24:57.972622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d24993-70c3-4b06-8d09-c8587a37c25d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:57.977996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-253af47a-de26-4341-b116-9421a9832e31 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:57.979186 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-253af47a-de26-4341-b116-9421a9832e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:58.038873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:58.064140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 529/1057] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:24:58.085080 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d24993-70c3-4b06-8d09-c8587a37c25d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:58.085372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253af47a-de26-4341-b116-9421a9832e31 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:58.127856 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091aa499-d19e-4908-b704-7aa1b552db97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:58.128522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091aa499-d19e-4908-b704-7aa1b552db97 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:58.129111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091aa499-d19e-4908-b704-7aa1b552db97 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:58.129388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091aa499-d19e-4908-b704-7aa1b552db97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:58.136232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091aa499-d19e-4908-b704-7aa1b552db97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:58.137667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091aa499-d19e-4908-b704-7aa1b552db97 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:58.140484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-091aa499-d19e-4908-b704-7aa1b552db97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:24:58.150043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:58.155215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:58.156198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:24:58.156198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:24:58.156198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:24:58.156198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:24:58.156198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:24:58.156198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:24:58.156198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:24:58.156198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:24:58.156198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 529/1058] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:24:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:58.157213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091aa499-d19e-4908-b704-7aa1b552db97 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:58.193470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253af47a-de26-4341-b116-9421a9832e31 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:58.229712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa3636-07e5-4689-b78e-2a464065f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:58.230353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa3636-07e5-4689-b78e-2a464065f80d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:58.230631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa3636-07e5-4689-b78e-2a464065f80d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:58.231151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa3636-07e5-4689-b78e-2a464065f80d None admin] PATH_INFO: `/v3/services/image` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:58.231774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa3636-07e5-4689-b78e-2a464065f80d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:58.233189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa3636-07e5-4689-b78e-2a464065f80d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c9fa3636-07e5-4689-b78e-2a464065f80d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:58.242429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:58.243468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:58.244936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 530/1059] 10.222.0.64 () {56 vars in 1105 bytes} [Thu Aug 4 07:24:58 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:24:58.249614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa3636-07e5-4689-b78e-2a464065f80d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:58.267755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091aa499-d19e-4908-b704-7aa1b552db97 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:58.301970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:58.303900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8c8b-72e4-4b00-917a-d344fcaf8c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:58.304553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8c8b-72e4-4b00-917a-d344fcaf8c37 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:58.304837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8c8b-72e4-4b00-917a-d344fcaf8c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:58.305115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8c8b-72e4-4b00-917a-d344fcaf8c37 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:58.305877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8c8b-72e4-4b00-917a-d344fcaf8c37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:58.306193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8c8b-72e4-4b00-917a-d344fcaf8c37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:58.307627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8c8b-72e4-4b00-917a-d344fcaf8c37 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:58.320070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 530/1060] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:24:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:58.325936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8c8b-72e4-4b00-917a-d344fcaf8c37 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:58.360226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa3636-07e5-4689-b78e-2a464065f80d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:58.390448 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:58.396263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f57122-5bea-458a-a614-1ad50adf09db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:58.396885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f57122-5bea-458a-a614-1ad50adf09db None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:58.397170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f57122-5bea-458a-a614-1ad50adf09db None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:58.397542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f57122-5bea-458a-a614-1ad50adf09db None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:58.398122 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f57122-5bea-458a-a614-1ad50adf09db None admin] RBAC: Adding query filter params (name=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:58.398776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f57122-5bea-458a-a614-1ad50adf09db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:58.399799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f57122-5bea-458a-a614-1ad50adf09db None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:58.410435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 531/1061] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:58.411305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f57122-5bea-458a-a614-1ad50adf09db None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:58.434467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8c8b-72e4-4b00-917a-d344fcaf8c37 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:58.476252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc390a-9bee-4e11-a7d8-c77da27faca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:58.477116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc390a-9bee-4e11-a7d8-c77da27faca8 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:58.477690 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc390a-9bee-4e11-a7d8-c77da27faca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:58.478071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc390a-9bee-4e11-a7d8-c77da27faca8 None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/4eedd711c6114de1bfcdf692ce058794/roles/eefaec6a74bd4d5c93eca0ae44a177fa` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:58.480680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc390a-9bee-4e11-a7d8-c77da27faca8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=4eedd711c6114de1bfcdf692ce058794, role_id=eefaec6a74bd4d5c93eca0ae44a177fa)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:58.483058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc390a-9bee-4e11-a7d8-c77da27faca8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:58.516562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f57122-5bea-458a-a614-1ad50adf09db None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:58.528924 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:58.549853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2737d10-fd17-447f-8b3a-0c48b19676c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:58.550474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737d10-fd17-447f-8b3a-0c48b19676c1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:58.550756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737d10-fd17-447f-8b3a-0c48b19676c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:58.551127 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737d10-fd17-447f-8b3a-0c48b19676c1 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:58.551798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2737d10-fd17-447f-8b3a-0c48b19676c1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:24:58.552184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2737d10-fd17-447f-8b3a-0c48b19676c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:58.553512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2737d10-fd17-447f-8b3a-0c48b19676c1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:58.554753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 531/1062] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:24:58 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/4eedd711c6114de1bfcdf692ce058794/roles/eefaec6a74bd4d5c93eca0ae44a177fa => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:24:58.560225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 532/1063] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:24:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:24:58.566743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc390a-9bee-4e11-a7d8-c77da27faca8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:58.679537 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc390a-9bee-4e11-a7d8-c77da27faca8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:58.701241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:24:58.720141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b628982-fd25-49a1-aadd-c54974c3e3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:58.720792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b628982-fd25-49a1-aadd-c54974c3e3d4 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:58.721078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b628982-fd25-49a1-aadd-c54974c3e3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:58.721353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b628982-fd25-49a1-aadd-c54974c3e3d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:58.722091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b628982-fd25-49a1-aadd-c54974c3e3d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:24:58.723567 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b628982-fd25-49a1-aadd-c54974c3e3d4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:24:58.823343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 532/1064] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:24:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:24:59.635473 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40bc10-da3b-4d1f-8319-2697c8df20e6 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:59.636068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40bc10-da3b-4d1f-8319-2697c8df20e6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:59.636068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40bc10-da3b-4d1f-8319-2697c8df20e6 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:59.637416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 533/1065] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:24:59.642722 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d8134-027c-4b3d-a90f-302a5aa41d1d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:59.643124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d8134-027c-4b3d-a90f-302a5aa41d1d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:59.643316 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d8134-027c-4b3d-a90f-302a5aa41d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:59.661704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ac5d8134-027c-4b3d-a90f-302a5aa41d1d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:24:59.662894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ac5d8134-027c-4b3d-a90f-302a5aa41d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:24:59.737635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 533/1066] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:24:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:24:59.760760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40bc10-da3b-4d1f-8319-2697c8df20e6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:24:59.862656 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40bc10-da3b-4d1f-8319-2697c8df20e6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:24:59.896245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a046ed-9446-46d4-9360-487e56fad406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:24:59.896808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a046ed-9446-46d4-9360-487e56fad406 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:24:59.897089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a046ed-9446-46d4-9360-487e56fad406 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:24:59.897366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a046ed-9446-46d4-9360-487e56fad406 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:24:59.908330 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a046ed-9446-46d4-9360-487e56fad406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d0a046ed-9446-46d4-9360-487e56fad406 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:24:59.913286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:24:59.914212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:24:59.915060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:24:59.919462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 534/1067] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:24:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:24:59.920712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d8134-027c-4b3d-a90f-302a5aa41d1d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:00.027042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d8134-027c-4b3d-a90f-302a5aa41d1d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:00.028581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:00.060962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbeef8-45ed-40ab-b2ab-3e5af3c8db68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:00.061580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbeef8-45ed-40ab-b2ab-3e5af3c8db68 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:00.061859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbeef8-45ed-40ab-b2ab-3e5af3c8db68 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:00.062159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbeef8-45ed-40ab-b2ab-3e5af3c8db68 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:00.063623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbeef8-45ed-40ab-b2ab-3e5af3c8db68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:00.064260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbeef8-45ed-40ab-b2ab-3e5af3c8db68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:00.066508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbeef8-45ed-40ab-b2ab-3e5af3c8db68 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:00.078859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 534/1068] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:24:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:00.081430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a046ed-9446-46d4-9360-487e56fad406 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:00.116323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:00.180335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a046ed-9446-46d4-9360-487e56fad406 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:00.215489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25935f59-d3fe-4b42-9a3f-d7081f452c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:00.217691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25935f59-d3fe-4b42-9a3f-d7081f452c37 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:00.217691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25935f59-d3fe-4b42-9a3f-d7081f452c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:00.217691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25935f59-d3fe-4b42-9a3f-d7081f452c37 None admin] PATH_INFO: `/v3/users/e05967a45e7b4c99bb9c3b63c8351b56` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:00.229881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25935f59-d3fe-4b42-9a3f-d7081f452c37 None admin] RBAC: Authorizing `identity:get_user(user_id=e05967a45e7b4c99bb9c3b63c8351b56)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:00.231564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25935f59-d3fe-4b42-9a3f-d7081f452c37 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:00.233397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 535/1069] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:00 2022] GET /identity/v3/users/e05967a45e7b4c99bb9c3b63c8351b56 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:00.239006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbeef8-45ed-40ab-b2ab-3e5af3c8db68 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:00.338351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbeef8-45ed-40ab-b2ab-3e5af3c8db68 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:00.371464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2012c9-798d-42f7-840a-9e86e6bf8e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:00.372104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2012c9-798d-42f7-840a-9e86e6bf8e35 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:00.372388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2012c9-798d-42f7-840a-9e86e6bf8e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:00.372661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2012c9-798d-42f7-840a-9e86e6bf8e35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:00.403622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:00.441003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2012c9-798d-42f7-840a-9e86e6bf8e35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:00.442369 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2012c9-798d-42f7-840a-9e86e6bf8e35 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:00.543967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 535/1070] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:25:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:25:00.545216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25935f59-d3fe-4b42-9a3f-d7081f452c37 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:00.587625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238c716-60f2-4abe-827c-10f8a1826622 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:00.587951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238c716-60f2-4abe-827c-10f8a1826622 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:00.588280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238c716-60f2-4abe-827c-10f8a1826622 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:00.589686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 536/1071] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:25:00.594514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64817-3dd5-4d43-8e5e-23e816ba495d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:00.594807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64817-3dd5-4d43-8e5e-23e816ba495d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:00.595091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64817-3dd5-4d43-8e5e-23e816ba495d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:00.615949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-43e64817-3dd5-4d43-8e5e-23e816ba495d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:00.617168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-43e64817-3dd5-4d43-8e5e-23e816ba495d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:00.641044 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:00.649526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25935f59-d3fe-4b42-9a3f-d7081f452c37 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:00.679757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:00.683919 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0159f73-f1f2-4822-80ae-0b5d56609bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:00.684623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0159f73-f1f2-4822-80ae-0b5d56609bcb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:00.684897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0159f73-f1f2-4822-80ae-0b5d56609bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:00.685196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0159f73-f1f2-4822-80ae-0b5d56609bcb None admin] PATH_INFO: `/v3/projects/5445df38221b45d49ff15ad288d79d68` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:00.686331 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0159f73-f1f2-4822-80ae-0b5d56609bcb None admin] RBAC: Authorizing `identity:get_project(project_id=5445df38221b45d49ff15ad288d79d68)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:00.688699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0159f73-f1f2-4822-80ae-0b5d56609bcb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:00.690361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 536/1072] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:00 2022] GET /identity/v3/projects/5445df38221b45d49ff15ad288d79d68 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:25:00.690701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 537/1073] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:25:00.696308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0159f73-f1f2-4822-80ae-0b5d56609bcb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:00.714814 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e64817-3dd5-4d43-8e5e-23e816ba495d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:00.800684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0159f73-f1f2-4822-80ae-0b5d56609bcb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:00.829543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e64817-3dd5-4d43-8e5e-23e816ba495d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:00.838706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26548f6-88a0-461e-a88b-5940c79527f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:00.839323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26548f6-88a0-461e-a88b-5940c79527f1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:00.839604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26548f6-88a0-461e-a88b-5940c79527f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:00.839886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26548f6-88a0-461e-a88b-5940c79527f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:00.840700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26548f6-88a0-461e-a88b-5940c79527f1 None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=5445df38221b45d49ff15ad288d79d68, user.id=e05967a45e7b4c99bb9c3b63c8351b56) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:00.841013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26548f6-88a0-461e-a88b-5940c79527f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:00.842439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26548f6-88a0-461e-a88b-5940c79527f1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:00.848374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 537/1074] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:25:00 2022] GET /identity/v3/role_assignments?user.id=e05967a45e7b4c99bb9c3b63c8351b56&scope.project.id=5445df38221b45d49ff15ad288d79d68&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:25:00.868731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700b388-b443-47bd-b0f1-281496389096 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:00.869015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700b388-b443-47bd-b0f1-281496389096 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:00.869293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700b388-b443-47bd-b0f1-281496389096 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:00.870841 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 538/1075] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:25:00.877530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b148-d26c-4077-bf57-c00368f4e56e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:00.877856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b148-d26c-4077-bf57-c00368f4e56e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:00.878086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b148-d26c-4077-bf57-c00368f4e56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:00.879152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e9b30-dc09-4736-9c01-8ac29a51aa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:00.879810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e9b30-dc09-4736-9c01-8ac29a51aa90 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:00.880133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e9b30-dc09-4736-9c01-8ac29a51aa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:00.880408 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e9b30-dc09-4736-9c01-8ac29a51aa90 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:00.893251 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e9b30-dc09-4736-9c01-8ac29a51aa90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:25:00.897288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9ec0b148-d26c-4077-bf57-c00368f4e56e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-bc5e9b30-dc09-4736-9c01-8ac29a51aa90 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:25:00.898190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:00.899096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 07:25:00.904978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:25:00.905627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9ec0b148-d26c-4077-bf57-c00368f4e56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:00.905704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 538/1076] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Aug 4 07:25:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:00.909042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e9b30-dc09-4736-9c01-8ac29a51aa90 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:00.924780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:00.980194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:00.982158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 539/1077] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:25:01.005550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0b148-d26c-4077-bf57-c00368f4e56e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:01.029594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e9b30-dc09-4736-9c01-8ac29a51aa90 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:01.065481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0507637-e71f-4317-bf8b-e4346ccb31b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:01.066098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0507637-e71f-4317-bf8b-e4346ccb31b4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:01.066384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0507637-e71f-4317-bf8b-e4346ccb31b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:01.066662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0507637-e71f-4317-bf8b-e4346ccb31b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:01.067556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0507637-e71f-4317-bf8b-e4346ccb31b4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:01.067879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0507637-e71f-4317-bf8b-e4346ccb31b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:01.069278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0507637-e71f-4317-bf8b-e4346ccb31b4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:01.077653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 539/1078] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Aug 4 07:25:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:01.083259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0507637-e71f-4317-bf8b-e4346ccb31b4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:01.113185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0b148-d26c-4077-bf57-c00368f4e56e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:01.149171 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead49a5-d62b-4dfb-b617-3aa2b92e4442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:01.149902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead49a5-d62b-4dfb-b617-3aa2b92e4442 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:01.150188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead49a5-d62b-4dfb-b617-3aa2b92e4442 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:01.150496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead49a5-d62b-4dfb-b617-3aa2b92e4442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:01.178857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:01.187686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0507637-e71f-4317-bf8b-e4346ccb31b4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:01.206744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:01.225289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead49a5-d62b-4dfb-b617-3aa2b92e4442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:01.226204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3875773-01b2-4124-8693-457fd9139104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:01.226676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead49a5-d62b-4dfb-b617-3aa2b92e4442 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:01.226863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3875773-01b2-4124-8693-457fd9139104 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:01.227131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3875773-01b2-4124-8693-457fd9139104 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:01.227406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3875773-01b2-4124-8693-457fd9139104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:01.239563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3875773-01b2-4124-8693-457fd9139104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:01.241341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3875773-01b2-4124-8693-457fd9139104 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d3875773-01b2-4124-8693-457fd9139104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:25:01.257846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:01.264225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:01.265352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:25:01.265352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:25:01.265352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:25:01.265352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:25:01.266320 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 540/1079] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:01.267689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3875773-01b2-4124-8693-457fd9139104 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:01.338088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 540/1080] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:25:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 07:25:01.343941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead49a5-d62b-4dfb-b617-3aa2b92e4442 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:01.373912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3875773-01b2-4124-8693-457fd9139104 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:01.410745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac190f6-2f55-4651-ba40-66870ff7ef72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:01.411367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac190f6-2f55-4651-ba40-66870ff7ef72 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:01.411653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac190f6-2f55-4651-ba40-66870ff7ef72 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:01.411932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac190f6-2f55-4651-ba40-66870ff7ef72 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:01.412661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac190f6-2f55-4651-ba40-66870ff7ef72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:01.412971 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac190f6-2f55-4651-ba40-66870ff7ef72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:01.414336 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac190f6-2f55-4651-ba40-66870ff7ef72 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:01.415835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:01.427261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 541/1081] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:01.432857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac190f6-2f55-4651-ba40-66870ff7ef72 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:01.461673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead49a5-d62b-4dfb-b617-3aa2b92e4442 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:01.462741 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:01.498927 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6be90-1bfd-4f5b-b98a-f1d95b2e0cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:01.499915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6be90-1bfd-4f5b-b98a-f1d95b2e0cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:01.500412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6be90-1bfd-4f5b-b98a-f1d95b2e0cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:01.500822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6be90-1bfd-4f5b-b98a-f1d95b2e0cc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:01.512450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6be90-1bfd-4f5b-b98a-f1d95b2e0cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:01.514067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6be90-1bfd-4f5b-b98a-f1d95b2e0cc6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-edf6be90-1bfd-4f5b-b98a-f1d95b2e0cc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:25:01.529463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:01.530600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:01.532280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:25:01.532280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:25:01.532280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:25:01.532280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:25:01.534116 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 541/1082] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:01.539690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6be90-1bfd-4f5b-b98a-f1d95b2e0cc6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:01.540119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac190f6-2f55-4651-ba40-66870ff7ef72 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:01.576460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2853c1-3dfa-4695-b09b-fc1d679777b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:01.577104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2853c1-3dfa-4695-b09b-fc1d679777b3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:01.577401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2853c1-3dfa-4695-b09b-fc1d679777b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:01.577665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2853c1-3dfa-4695-b09b-fc1d679777b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:01.585735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2853c1-3dfa-4695-b09b-fc1d679777b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:01.587175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2853c1-3dfa-4695-b09b-fc1d679777b3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7a2853c1-3dfa-4695-b09b-fc1d679777b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:25:01.599174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:01.600352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:01.601896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:01.601896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:01.601896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:25:01.601896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:25:01.601896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:25:01.601896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:25:01.601896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:25:01.601896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:25:01.601896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 07:25:01.601896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:25:01.601896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 542/1083] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Aug 4 07:25:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:25:01.606230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2853c1-3dfa-4695-b09b-fc1d679777b3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:01.652246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6be90-1bfd-4f5b-b98a-f1d95b2e0cc6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:01.661590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:01.687109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9648e-b69b-4500-b449-20aad78e2a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:01.688402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9648e-b69b-4500-b449-20aad78e2a5e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:01.688686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9648e-b69b-4500-b449-20aad78e2a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:01.688961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9648e-b69b-4500-b449-20aad78e2a5e None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:01.689655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9648e-b69b-4500-b449-20aad78e2a5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:01.690016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9648e-b69b-4500-b449-20aad78e2a5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:01.691485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9648e-b69b-4500-b449-20aad78e2a5e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:01.697591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:01.706183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 542/1084] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:01.710374 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2853c1-3dfa-4695-b09b-fc1d679777b3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:01.749558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d73c6-c263-4a2e-87b1-37b8ae69372a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:01.750640 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d73c6-c263-4a2e-87b1-37b8ae69372a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:01.750926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d73c6-c263-4a2e-87b1-37b8ae69372a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:01.751357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d73c6-c263-4a2e-87b1-37b8ae69372a None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:01.752212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d73c6-c263-4a2e-87b1-37b8ae69372a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:01.752527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d73c6-c263-4a2e-87b1-37b8ae69372a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:01.754009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d73c6-c263-4a2e-87b1-37b8ae69372a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:01.779180 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 543/1085] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Aug 4 07:25:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:25:01.780620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9648e-b69b-4500-b449-20aad78e2a5e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:01.891787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9648e-b69b-4500-b449-20aad78e2a5e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:01.928233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1c935-0b3b-4518-a510-50c2a8106de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:01.928862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1c935-0b3b-4518-a510-50c2a8106de2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:01.929144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1c935-0b3b-4518-a510-50c2a8106de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:01.929419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1c935-0b3b-4518-a510-50c2a8106de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:01.977041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:02.012583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1c935-0b3b-4518-a510-50c2a8106de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:02.013969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1c935-0b3b-4518-a510-50c2a8106de2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:02.110481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 543/1086] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:25:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:25:02.116291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d73c6-c263-4a2e-87b1-37b8ae69372a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:02.211561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d73c6-c263-4a2e-87b1-37b8ae69372a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:02.245426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5785234-fca0-4965-b560-e72ec642cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:02.246170 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5785234-fca0-4965-b560-e72ec642cc42 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:02.246453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5785234-fca0-4965-b560-e72ec642cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:02.246732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5785234-fca0-4965-b560-e72ec642cc42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:02.256761 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5785234-fca0-4965-b560-e72ec642cc42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:02.258198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5785234-fca0-4965-b560-e72ec642cc42 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:02.264875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e5785234-fca0-4965-b560-e72ec642cc42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:25:02.274150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:02.281107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:02.282966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:25:02.282966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:25:02.282966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:25:02.282966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:25:02.282966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 544/1087] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:02.283304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1c935-0b3b-4518-a510-50c2a8106de2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:02.348680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:02.381871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1c935-0b3b-4518-a510-50c2a8106de2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:02.414540 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac25d8-29aa-48ca-80e0-1bc6812bf7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:02.415151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac25d8-29aa-48ca-80e0-1bc6812bf7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:02.415432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac25d8-29aa-48ca-80e0-1bc6812bf7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:02.415705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac25d8-29aa-48ca-80e0-1bc6812bf7c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:02.416421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac25d8-29aa-48ca-80e0-1bc6812bf7c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:02.416728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac25d8-29aa-48ca-80e0-1bc6812bf7c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:02.418089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac25d8-29aa-48ca-80e0-1bc6812bf7c9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:02.429682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 544/1088] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:02.435267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5785234-fca0-4965-b560-e72ec642cc42 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:02.532740 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5785234-fca0-4965-b560-e72ec642cc42 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:02.565020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d223da-cfd2-439c-8221-e727b792221d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:02.565657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d223da-cfd2-439c-8221-e727b792221d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:02.565939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d223da-cfd2-439c-8221-e727b792221d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:02.566214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d223da-cfd2-439c-8221-e727b792221d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:02.572768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d223da-cfd2-439c-8221-e727b792221d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:02.574188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d223da-cfd2-439c-8221-e727b792221d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-87d223da-cfd2-439c-8221-e727b792221d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:25:02.586842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:02.587885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:02.593474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:25:02.593474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:25:02.593474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:25:02.593474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:25:02.593474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:25:02.593474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:25:02.593474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:25:02.593474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:25:02.593474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 545/1089] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:25:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:02.593998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac25d8-29aa-48ca-80e0-1bc6812bf7c9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:02.692966 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac25d8-29aa-48ca-80e0-1bc6812bf7c9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:02.718805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:02.726457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545ab9e-ee94-449e-9096-23c59c0f89fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:02.727052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545ab9e-ee94-449e-9096-23c59c0f89fc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:02.727332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545ab9e-ee94-449e-9096-23c59c0f89fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:02.727605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545ab9e-ee94-449e-9096-23c59c0f89fc None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:02.728385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545ab9e-ee94-449e-9096-23c59c0f89fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:02.728689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545ab9e-ee94-449e-9096-23c59c0f89fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:02.730120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545ab9e-ee94-449e-9096-23c59c0f89fc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:02.742184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 545/1090] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:25:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:25:02.748129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d223da-cfd2-439c-8221-e727b792221d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:02.805710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:02.847356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d223da-cfd2-439c-8221-e727b792221d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:02.873471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303d7fc-71c3-4a6a-9ad3-08c05c85bfe5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:02.873845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303d7fc-71c3-4a6a-9ad3-08c05c85bfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:02.874189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303d7fc-71c3-4a6a-9ad3-08c05c85bfe5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:02.876623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 546/1091] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:25:02.881575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd616fb4-51c4-4755-a2f8-eec119c487cc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:02.881853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd616fb4-51c4-4755-a2f8-eec119c487cc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:02.881953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5870ed-d453-4e5c-802a-b3d0f48884bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:02.882127 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd616fb4-51c4-4755-a2f8-eec119c487cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:02.882483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5870ed-d453-4e5c-802a-b3d0f48884bd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:02.882766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5870ed-d453-4e5c-802a-b3d0f48884bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:02.883043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5870ed-d453-4e5c-802a-b3d0f48884bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:02.883839 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5870ed-d453-4e5c-802a-b3d0f48884bd None admin] RBAC: Adding query filter params (role.id=eefaec6a74bd4d5c93eca0ae44a177fa, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=4eedd711c6114de1bfcdf692ce058794) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:02.884188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5870ed-d453-4e5c-802a-b3d0f48884bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:02.885600 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5870ed-d453-4e5c-802a-b3d0f48884bd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:02.891744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 546/1092] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:25:02 2022] GET /identity/v3/role_assignments?user.id=4eedd711c6114de1bfcdf692ce058794&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=eefaec6a74bd4d5c93eca0ae44a177fa => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:02.900641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cd616fb4-51c4-4755-a2f8-eec119c487cc None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:02.901888 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cd616fb4-51c4-4755-a2f8-eec119c487cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:02.975596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 547/1093] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:25:02.999073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5870ed-d453-4e5c-802a-b3d0f48884bd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:03.102205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5870ed-d453-4e5c-802a-b3d0f48884bd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:03.125308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:03.137017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298059-76d8-4dda-a37a-8f0d9577c62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:03.137680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298059-76d8-4dda-a37a-8f0d9577c62b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:03.137967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298059-76d8-4dda-a37a-8f0d9577c62b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:03.138247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298059-76d8-4dda-a37a-8f0d9577c62b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:03.148945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de298059-76d8-4dda-a37a-8f0d9577c62b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-de298059-76d8-4dda-a37a-8f0d9577c62b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:25:03.153710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:03.154661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:03.155514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:25:03.160146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 547/1094] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:25:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:03.161758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd616fb4-51c4-4755-a2f8-eec119c487cc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:03.268327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd616fb4-51c4-4755-a2f8-eec119c487cc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:03.286364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:03.305382 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fbf4-beae-4919-9be5-b654f60b5715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:03.305998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fbf4-beae-4919-9be5-b654f60b5715 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:03.306280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fbf4-beae-4919-9be5-b654f60b5715 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:03.306555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fbf4-beae-4919-9be5-b654f60b5715 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:03.307275 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2fbf4-beae-4919-9be5-b654f60b5715 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:03.307692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2fbf4-beae-4919-9be5-b654f60b5715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:03.309464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2fbf4-beae-4919-9be5-b654f60b5715 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:03.318505 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 548/1095] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:25:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:03.324704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298059-76d8-4dda-a37a-8f0d9577c62b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:03.430377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298059-76d8-4dda-a37a-8f0d9577c62b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:03.464308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10822e07-cc07-4c46-8ec1-ab3aacc44862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:03.464952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822e07-cc07-4c46-8ec1-ab3aacc44862 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:03.465236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822e07-cc07-4c46-8ec1-ab3aacc44862 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:03.465518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822e07-cc07-4c46-8ec1-ab3aacc44862 None admin] PATH_INFO: `/v3/users/e05967a45e7b4c99bb9c3b63c8351b56` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:03.466678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10822e07-cc07-4c46-8ec1-ab3aacc44862 None admin] RBAC: Authorizing `identity:get_user(user_id=e05967a45e7b4c99bb9c3b63c8351b56)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:03.468156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10822e07-cc07-4c46-8ec1-ab3aacc44862 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:03.470598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 548/1096] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:03 2022] GET /identity/v3/users/e05967a45e7b4c99bb9c3b63c8351b56 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:03.481106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fbf4-beae-4919-9be5-b654f60b5715 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:03.581197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fbf4-beae-4919-9be5-b654f60b5715 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:03.616589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246860e9-94b9-42fd-875c-f209e3b8b2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:03.617222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246860e9-94b9-42fd-875c-f209e3b8b2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:03.617505 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246860e9-94b9-42fd-875c-f209e3b8b2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:03.617780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246860e9-94b9-42fd-875c-f209e3b8b2a9 None admin] PATH_INFO: `/v3/projects/5445df38221b45d49ff15ad288d79d68` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:03.618937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246860e9-94b9-42fd-875c-f209e3b8b2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=5445df38221b45d49ff15ad288d79d68)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:03.620390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246860e9-94b9-42fd-875c-f209e3b8b2a9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:03.622249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 549/1097] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:03 2022] GET /identity/v3/projects/5445df38221b45d49ff15ad288d79d68 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:03.628168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10822e07-cc07-4c46-8ec1-ab3aacc44862 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:03.680939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:03.732655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10822e07-cc07-4c46-8ec1-ab3aacc44862 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:03.765462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9320528-67e3-494a-9b9a-389f48eddf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:03.766795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9320528-67e3-494a-9b9a-389f48eddf51 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:03.767082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9320528-67e3-494a-9b9a-389f48eddf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:03.767357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9320528-67e3-494a-9b9a-389f48eddf51 None admin] PATH_INFO: `/v3/projects/5445df38221b45d49ff15ad288d79d68/users/e05967a45e7b4c99bb9c3b63c8351b56/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:03.769307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9320528-67e3-494a-9b9a-389f48eddf51 None admin] RBAC: Authorizing `identity:create_grant(project_id=5445df38221b45d49ff15ad288d79d68, user_id=e05967a45e7b4c99bb9c3b63c8351b56, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:03.770732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9320528-67e3-494a-9b9a-389f48eddf51 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:03.817850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 549/1098] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:25:03 2022] PUT /identity/v3/projects/5445df38221b45d49ff15ad288d79d68/users/e05967a45e7b4c99bb9c3b63c8351b56/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:25:05.786246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a8c28-034d-4ab9-94c1-dba66f637d20 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:05.786819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a8c28-034d-4ab9-94c1-dba66f637d20 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:05.786819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a8c28-034d-4ab9-94c1-dba66f637d20 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:05.788181 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 550/1099] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:05.793339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88858-0862-4109-a903-543c4ef63faa None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:05.793776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88858-0862-4109-a903-543c4ef63faa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:05.793878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88858-0862-4109-a903-543c4ef63faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:05.811441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8de88858-0862-4109-a903-543c4ef63faa None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:05.812688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8de88858-0862-4109-a903-543c4ef63faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:05.876897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 550/1100] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:05.900546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a8c28-034d-4ab9-94c1-dba66f637d20 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:05.950350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:06.003402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a8c28-034d-4ab9-94c1-dba66f637d20 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:06.034571 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87772d2d-f491-4885-942d-554accdf4f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:06.035189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87772d2d-f491-4885-942d-554accdf4f9d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:06.035465 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87772d2d-f491-4885-942d-554accdf4f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:06.035733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87772d2d-f491-4885-942d-554accdf4f9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:06.045728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87772d2d-f491-4885-942d-554accdf4f9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-87772d2d-f491-4885-942d-554accdf4f9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:25:06.050981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:06.052142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:25:06.053002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 551/1101] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:25:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:06.058235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de88858-0862-4109-a903-543c4ef63faa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:06.099513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:06.154503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de88858-0862-4109-a903-543c4ef63faa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:06.188872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee6f1f-6d82-44f1-a111-27a3a46f5e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:06.189483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee6f1f-6d82-44f1-a111-27a3a46f5e99 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:06.189768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee6f1f-6d82-44f1-a111-27a3a46f5e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:06.190045 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee6f1f-6d82-44f1-a111-27a3a46f5e99 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:06.190758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee6f1f-6d82-44f1-a111-27a3a46f5e99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:06.191066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee6f1f-6d82-44f1-a111-27a3a46f5e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:06.192474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee6f1f-6d82-44f1-a111-27a3a46f5e99 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:06.200319 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 551/1102] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:25:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:06.206056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87772d2d-f491-4885-942d-554accdf4f9d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:06.303505 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87772d2d-f491-4885-942d-554accdf4f9d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:06.311968 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:06.335934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb41647-4e31-41ea-9850-4293f9724726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:06.336602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb41647-4e31-41ea-9850-4293f9724726 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:06.336886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb41647-4e31-41ea-9850-4293f9724726 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:06.337165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb41647-4e31-41ea-9850-4293f9724726 None admin] PATH_INFO: `/v3/users/e05967a45e7b4c99bb9c3b63c8351b56` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:06.338314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb41647-4e31-41ea-9850-4293f9724726 None admin] RBAC: Authorizing `identity:get_user(user_id=e05967a45e7b4c99bb9c3b63c8351b56)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:06.339746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb41647-4e31-41ea-9850-4293f9724726 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:06.341363 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 552/1103] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:06 2022] GET /identity/v3/users/e05967a45e7b4c99bb9c3b63c8351b56 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:06.347793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee6f1f-6d82-44f1-a111-27a3a46f5e99 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:06.445273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee6f1f-6d82-44f1-a111-27a3a46f5e99 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:06.446728 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:06.478407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a507d7-9a0b-4579-a0fa-0cc9c6c7a9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:06.479008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a507d7-9a0b-4579-a0fa-0cc9c6c7a9bc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:06.479287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a507d7-9a0b-4579-a0fa-0cc9c6c7a9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:06.479561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a507d7-9a0b-4579-a0fa-0cc9c6c7a9bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:06.544423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a507d7-9a0b-4579-a0fa-0cc9c6c7a9bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:06.545850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a507d7-9a0b-4579-a0fa-0cc9c6c7a9bc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:06.639213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 552/1104] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:25:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:25:06.645693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb41647-4e31-41ea-9850-4293f9724726 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:06.742931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb41647-4e31-41ea-9850-4293f9724726 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:06.775141 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f325b12-b2c6-4c93-bbf9-78d1f54b64d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:06.775763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f325b12-b2c6-4c93-bbf9-78d1f54b64d9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:06.776075 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f325b12-b2c6-4c93-bbf9-78d1f54b64d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:06.776356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f325b12-b2c6-4c93-bbf9-78d1f54b64d9 None admin] PATH_INFO: `/v3/projects/5445df38221b45d49ff15ad288d79d68` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:06.777490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f325b12-b2c6-4c93-bbf9-78d1f54b64d9 None admin] RBAC: Authorizing `identity:get_project(project_id=5445df38221b45d49ff15ad288d79d68)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:06.778910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f325b12-b2c6-4c93-bbf9-78d1f54b64d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:06.780559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 553/1105] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:06 2022] GET /identity/v3/projects/5445df38221b45d49ff15ad288d79d68 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:06.787378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a507d7-9a0b-4579-a0fa-0cc9c6c7a9bc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:06.789519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:06.884977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a507d7-9a0b-4579-a0fa-0cc9c6c7a9bc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:06.918532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48564467-575e-480c-bedf-c081eed4003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:06.919075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48564467-575e-480c-bedf-c081eed4003a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:06.919361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48564467-575e-480c-bedf-c081eed4003a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:06.919639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48564467-575e-480c-bedf-c081eed4003a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:06.920462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48564467-575e-480c-bedf-c081eed4003a None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=5445df38221b45d49ff15ad288d79d68, user.id=e05967a45e7b4c99bb9c3b63c8351b56) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:06.920777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48564467-575e-480c-bedf-c081eed4003a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:06.922222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48564467-575e-480c-bedf-c081eed4003a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:06.927851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 553/1106] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:25:06 2022] GET /identity/v3/role_assignments?user.id=e05967a45e7b4c99bb9c3b63c8351b56&scope.project.id=5445df38221b45d49ff15ad288d79d68&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:25:08.920181 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a2153-dfe7-482b-8c54-44eed5105a0e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:08.920712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a2153-dfe7-482b-8c54-44eed5105a0e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:08.920712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a2153-dfe7-482b-8c54-44eed5105a0e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:08.922896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 554/1107] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:08.927943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c4b69-2053-4b95-89d3-d3bf5d32c4ab None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:08.928366 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c4b69-2053-4b95-89d3-d3bf5d32c4ab None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:08.928531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c4b69-2053-4b95-89d3-d3bf5d32c4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:08.946616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-646c4b69-2053-4b95-89d3-d3bf5d32c4ab None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:08.947800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-646c4b69-2053-4b95-89d3-d3bf5d32c4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:09.004961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:09.011847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 554/1108] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:09.035618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a2153-dfe7-482b-8c54-44eed5105a0e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:09.082367 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:09.136504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a2153-dfe7-482b-8c54-44eed5105a0e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:09.168913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a3f48-32c1-42b7-8491-b68d74bb709b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:09.169562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a3f48-32c1-42b7-8491-b68d74bb709b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:09.169866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a3f48-32c1-42b7-8491-b68d74bb709b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:09.170167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a3f48-32c1-42b7-8491-b68d74bb709b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:09.171359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a3f48-32c1-42b7-8491-b68d74bb709b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:09.173436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a3f48-32c1-42b7-8491-b68d74bb709b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:09.175004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 555/1109] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:25:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:09.180664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c4b69-2053-4b95-89d3-d3bf5d32c4ab None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:09.275144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c4b69-2053-4b95-89d3-d3bf5d32c4ab None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:09.308493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04620c07-e3de-4b7c-a733-b7fc1829b63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:09.309090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04620c07-e3de-4b7c-a733-b7fc1829b63f None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:09.309369 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04620c07-e3de-4b7c-a733-b7fc1829b63f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:09.309641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04620c07-e3de-4b7c-a733-b7fc1829b63f None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:09.310314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04620c07-e3de-4b7c-a733-b7fc1829b63f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:09.311715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04620c07-e3de-4b7c-a733-b7fc1829b63f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:09.394223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 555/1110] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:25:09 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:25:11.398378 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392048a6-78b3-44d0-b6d6-1ab535a9a030 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:11.398898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392048a6-78b3-44d0-b6d6-1ab535a9a030 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:11.398898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392048a6-78b3-44d0-b6d6-1ab535a9a030 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:11.400206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 556/1111] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:11.405593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7dbb3-1871-4f44-b642-f7da15d783f0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:11.406006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7dbb3-1871-4f44-b642-f7da15d783f0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:11.406213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7dbb3-1871-4f44-b642-f7da15d783f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:11.430799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-25a7dbb3-1871-4f44-b642-f7da15d783f0 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:11.432009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-25a7dbb3-1871-4f44-b642-f7da15d783f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:11.477473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:11.496584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 556/1112] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:25:11.520671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392048a6-78b3-44d0-b6d6-1ab535a9a030 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:11.618281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392048a6-78b3-44d0-b6d6-1ab535a9a030 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:11.631210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:11.651931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7f9a1-dc5b-4d26-94c9-4325de0112c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:11.652528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7f9a1-dc5b-4d26-94c9-4325de0112c9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:11.652810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7f9a1-dc5b-4d26-94c9-4325de0112c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:11.653085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7f9a1-dc5b-4d26-94c9-4325de0112c9 None admin] PATH_INFO: `/v3/roles/e3c5a858a45c41db8ff3e883788d75bc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:11.654512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7f9a1-dc5b-4d26-94c9-4325de0112c9 None admin] RBAC: Authorizing `identity:get_role(role_id=e3c5a858a45c41db8ff3e883788d75bc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:11.655964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7f9a1-dc5b-4d26-94c9-4325de0112c9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:11.657287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 557/1113] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:11 2022] GET /identity/v3/roles/e3c5a858a45c41db8ff3e883788d75bc => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:11.663294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7dbb3-1871-4f44-b642-f7da15d783f0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:11.760294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7dbb3-1871-4f44-b642-f7da15d783f0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:11.792152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e926432-da59-46e0-a38a-4925ee7501ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:11.792774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e926432-da59-46e0-a38a-4925ee7501ed None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:11.793053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e926432-da59-46e0-a38a-4925ee7501ed None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:11.793327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e926432-da59-46e0-a38a-4925ee7501ed None admin] PATH_INFO: `/v3/users/ec820f1dd7ee43e1ae477769a85fab85` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:11.806381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e926432-da59-46e0-a38a-4925ee7501ed None admin] RBAC: Authorizing `identity:get_user(user_id=ec820f1dd7ee43e1ae477769a85fab85)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:11.807802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e926432-da59-46e0-a38a-4925ee7501ed None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:11.809420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 557/1114] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:11 2022] GET /identity/v3/users/ec820f1dd7ee43e1ae477769a85fab85 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:11.815862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7f9a1-dc5b-4d26-94c9-4325de0112c9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:11.914057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7f9a1-dc5b-4d26-94c9-4325de0112c9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:11.946735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede80be6-9084-4b18-a3db-b4c26b1c62b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:11.947334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede80be6-9084-4b18-a3db-b4c26b1c62b6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:11.947617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede80be6-9084-4b18-a3db-b4c26b1c62b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:11.947897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede80be6-9084-4b18-a3db-b4c26b1c62b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:11.995152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:12.014018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede80be6-9084-4b18-a3db-b4c26b1c62b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:12.015543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede80be6-9084-4b18-a3db-b4c26b1c62b6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:12.110039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 558/1115] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:25:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:25:12.116264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e926432-da59-46e0-a38a-4925ee7501ed None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:12.155360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:12.210552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e926432-da59-46e0-a38a-4925ee7501ed None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:12.244099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0518ed-fef3-4fe4-858c-1e524e9493c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:12.244764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0518ed-fef3-4fe4-858c-1e524e9493c6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:12.245046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0518ed-fef3-4fe4-858c-1e524e9493c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:12.245321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0518ed-fef3-4fe4-858c-1e524e9493c6 None admin] PATH_INFO: `/v3/projects/5445df38221b45d49ff15ad288d79d68` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:12.246451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0518ed-fef3-4fe4-858c-1e524e9493c6 None admin] RBAC: Authorizing `identity:get_project(project_id=5445df38221b45d49ff15ad288d79d68)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:12.247873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0518ed-fef3-4fe4-858c-1e524e9493c6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:12.249564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 558/1116] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:12 2022] GET /identity/v3/projects/5445df38221b45d49ff15ad288d79d68 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:12.255675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede80be6-9084-4b18-a3db-b4c26b1c62b6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:12.364395 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:12.382390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede80be6-9084-4b18-a3db-b4c26b1c62b6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:12.415542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137d7c0-33ce-47ff-9d1b-31aec2e9ddae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:12.416184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137d7c0-33ce-47ff-9d1b-31aec2e9ddae None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:12.416604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137d7c0-33ce-47ff-9d1b-31aec2e9ddae None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:12.416884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137d7c0-33ce-47ff-9d1b-31aec2e9ddae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:12.417663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137d7c0-33ce-47ff-9d1b-31aec2e9ddae None admin] RBAC: Adding query filter params (role.id=e3c5a858a45c41db8ff3e883788d75bc, scope.project.id=5445df38221b45d49ff15ad288d79d68, user.id=ec820f1dd7ee43e1ae477769a85fab85) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:12.417972 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137d7c0-33ce-47ff-9d1b-31aec2e9ddae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:12.419384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137d7c0-33ce-47ff-9d1b-31aec2e9ddae None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:12.425434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 559/1117] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:25:12 2022] GET /identity/v3/role_assignments?user.id=ec820f1dd7ee43e1ae477769a85fab85&scope.project.id=5445df38221b45d49ff15ad288d79d68&role.id=e3c5a858a45c41db8ff3e883788d75bc => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:14.392637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2074b-cc11-426e-8c94-59c42162a2ae None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:14.393139 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2074b-cc11-426e-8c94-59c42162a2ae None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:14.393139 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2074b-cc11-426e-8c94-59c42162a2ae None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:14.394453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 559/1118] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:14.399673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee9cb6-f6b7-4f60-b833-c8de4b6abf6b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:14.400115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee9cb6-f6b7-4f60-b833-c8de4b6abf6b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:14.400263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee9cb6-f6b7-4f60-b833-c8de4b6abf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:14.418415 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-58ee9cb6-f6b7-4f60-b833-c8de4b6abf6b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:14.419575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-58ee9cb6-f6b7-4f60-b833-c8de4b6abf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:14.482867 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 560/1119] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:14.505169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2074b-cc11-426e-8c94-59c42162a2ae None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:14.602947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2074b-cc11-426e-8c94-59c42162a2ae None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:14.613126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:14.636618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05d386-51d8-473a-9500-1e1cfe4dc500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:14.637240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05d386-51d8-473a-9500-1e1cfe4dc500 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:14.637523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05d386-51d8-473a-9500-1e1cfe4dc500 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:14.637801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05d386-51d8-473a-9500-1e1cfe4dc500 None admin] PATH_INFO: `/v3/roles/e3c5a858a45c41db8ff3e883788d75bc` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:14.639248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05d386-51d8-473a-9500-1e1cfe4dc500 None admin] RBAC: Authorizing `identity:get_role(role_id=e3c5a858a45c41db8ff3e883788d75bc)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:14.641068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05d386-51d8-473a-9500-1e1cfe4dc500 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:14.642706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 560/1120] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:14 2022] GET /identity/v3/roles/e3c5a858a45c41db8ff3e883788d75bc => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:14.648143 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee9cb6-f6b7-4f60-b833-c8de4b6abf6b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:14.736443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:14.755705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee9cb6-f6b7-4f60-b833-c8de4b6abf6b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:14.791395 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0961c86-cea3-4a3d-a620-98b993f3b6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:14.792182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0961c86-cea3-4a3d-a620-98b993f3b6ed None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:14.792532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0961c86-cea3-4a3d-a620-98b993f3b6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:14.792816 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0961c86-cea3-4a3d-a620-98b993f3b6ed None admin] PATH_INFO: `/v3/users/ec820f1dd7ee43e1ae477769a85fab85` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:14.793974 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0961c86-cea3-4a3d-a620-98b993f3b6ed None admin] RBAC: Authorizing `identity:get_user(user_id=ec820f1dd7ee43e1ae477769a85fab85)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:14.795416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0961c86-cea3-4a3d-a620-98b993f3b6ed None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:14.797174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 561/1121] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:14 2022] GET /identity/v3/users/ec820f1dd7ee43e1ae477769a85fab85 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:14.803105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05d386-51d8-473a-9500-1e1cfe4dc500 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:14.903472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05d386-51d8-473a-9500-1e1cfe4dc500 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:14.937293 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727486-e422-4730-b882-edc57e779d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:14.937933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727486-e422-4730-b882-edc57e779d85 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:14.938216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727486-e422-4730-b882-edc57e779d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:14.938493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727486-e422-4730-b882-edc57e779d85 None admin] PATH_INFO: `/v3/projects/5445df38221b45d49ff15ad288d79d68` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:14.939679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a727486-e422-4730-b882-edc57e779d85 None admin] RBAC: Authorizing `identity:get_project(project_id=5445df38221b45d49ff15ad288d79d68)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:14.941222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a727486-e422-4730-b882-edc57e779d85 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:14.942879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 561/1122] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:14 2022] GET /identity/v3/projects/5445df38221b45d49ff15ad288d79d68 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:14.949166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0961c86-cea3-4a3d-a620-98b993f3b6ed None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:15.049720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0961c86-cea3-4a3d-a620-98b993f3b6ed None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:15.082076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4dbd2-7c2e-4150-9552-df506ffa87ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:15.082729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4dbd2-7c2e-4150-9552-df506ffa87ee None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:15.083009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4dbd2-7c2e-4150-9552-df506ffa87ee None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:15.083283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4dbd2-7c2e-4150-9552-df506ffa87ee None admin] PATH_INFO: `/v3/projects/5445df38221b45d49ff15ad288d79d68/users/ec820f1dd7ee43e1ae477769a85fab85/roles/e3c5a858a45c41db8ff3e883788d75bc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:15.085429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4dbd2-7c2e-4150-9552-df506ffa87ee None admin] RBAC: Authorizing `identity:create_grant(project_id=5445df38221b45d49ff15ad288d79d68, user_id=ec820f1dd7ee43e1ae477769a85fab85, role_id=e3c5a858a45c41db8ff3e883788d75bc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:15.086855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4dbd2-7c2e-4150-9552-df506ffa87ee None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:15.154571 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 562/1123] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:25:14 2022] PUT /identity/v3/projects/5445df38221b45d49ff15ad288d79d68/users/ec820f1dd7ee43e1ae477769a85fab85/roles/e3c5a858a45c41db8ff3e883788d75bc => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:25:17.138275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f599e3-7a0b-42aa-a26d-3c1eeb62ab0b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:17.138851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f599e3-7a0b-42aa-a26d-3c1eeb62ab0b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:17.138851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f599e3-7a0b-42aa-a26d-3c1eeb62ab0b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:17.140698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 562/1124] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:17.145955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640952b1-d823-413f-97a6-bfab999b6fc7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:17.146375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640952b1-d823-413f-97a6-bfab999b6fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:17.146521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640952b1-d823-413f-97a6-bfab999b6fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:17.152810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:17.165991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-640952b1-d823-413f-97a6-bfab999b6fc7 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:17.167171 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-640952b1-d823-413f-97a6-bfab999b6fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:17.232899 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 563/1125] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:17.256476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f599e3-7a0b-42aa-a26d-3c1eeb62ab0b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:17.303421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:17.356746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f599e3-7a0b-42aa-a26d-3c1eeb62ab0b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:17.389242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dcdae-7a80-47cf-813e-70954feadd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:17.389901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dcdae-7a80-47cf-813e-70954feadd20 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:17.390307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dcdae-7a80-47cf-813e-70954feadd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:17.390586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dcdae-7a80-47cf-813e-70954feadd20 None admin] PATH_INFO: `/v3/roles/e3c5a858a45c41db8ff3e883788d75bc` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:17.392039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969dcdae-7a80-47cf-813e-70954feadd20 None admin] RBAC: Authorizing `identity:get_role(role_id=e3c5a858a45c41db8ff3e883788d75bc)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:17.393393 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969dcdae-7a80-47cf-813e-70954feadd20 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:17.394700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 563/1126] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:17 2022] GET /identity/v3/roles/e3c5a858a45c41db8ff3e883788d75bc => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:17.400420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640952b1-d823-413f-97a6-bfab999b6fc7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:17.494068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640952b1-d823-413f-97a6-bfab999b6fc7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:17.520949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:17.525626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e048ec0-a120-4c40-958b-96c95dea88de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:17.526246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e048ec0-a120-4c40-958b-96c95dea88de None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:17.526521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e048ec0-a120-4c40-958b-96c95dea88de None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:17.526789 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e048ec0-a120-4c40-958b-96c95dea88de None admin] PATH_INFO: `/v3/users/ec820f1dd7ee43e1ae477769a85fab85` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:17.527907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e048ec0-a120-4c40-958b-96c95dea88de None admin] RBAC: Authorizing `identity:get_user(user_id=ec820f1dd7ee43e1ae477769a85fab85)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:17.529348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e048ec0-a120-4c40-958b-96c95dea88de None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:17.531535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 564/1127] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:17 2022] GET /identity/v3/users/ec820f1dd7ee43e1ae477769a85fab85 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:17.537049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dcdae-7a80-47cf-813e-70954feadd20 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:17.633646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dcdae-7a80-47cf-813e-70954feadd20 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:17.646298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:17.667156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31389cc4-4789-4f0b-87d1-d93dfa72f938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:17.667763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31389cc4-4789-4f0b-87d1-d93dfa72f938 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:17.668077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31389cc4-4789-4f0b-87d1-d93dfa72f938 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:17.668357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31389cc4-4789-4f0b-87d1-d93dfa72f938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:17.734259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31389cc4-4789-4f0b-87d1-d93dfa72f938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:17.735697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31389cc4-4789-4f0b-87d1-d93dfa72f938 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:17.829875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 564/1128] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:25:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:25:17.836278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e048ec0-a120-4c40-958b-96c95dea88de None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:17.932767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e048ec0-a120-4c40-958b-96c95dea88de None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:17.965507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903ddc4-8100-48d6-b579-34b5d0ddda22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:17.966138 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ddc4-8100-48d6-b579-34b5d0ddda22 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:17.966420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ddc4-8100-48d6-b579-34b5d0ddda22 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:17.966695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ddc4-8100-48d6-b579-34b5d0ddda22 None admin] PATH_INFO: `/v3/projects/5445df38221b45d49ff15ad288d79d68` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:17.967832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903ddc4-8100-48d6-b579-34b5d0ddda22 None admin] RBAC: Authorizing `identity:get_project(project_id=5445df38221b45d49ff15ad288d79d68)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:17.969898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903ddc4-8100-48d6-b579-34b5d0ddda22 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:17.971547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 565/1129] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:17 2022] GET /identity/v3/projects/5445df38221b45d49ff15ad288d79d68 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:17.977603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31389cc4-4789-4f0b-87d1-d93dfa72f938 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:17.991547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:18.075310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31389cc4-4789-4f0b-87d1-d93dfa72f938 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:18.108789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedfd5b-a360-4831-bd4a-b976d10e964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:18.109347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedfd5b-a360-4831-bd4a-b976d10e964c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:18.109634 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedfd5b-a360-4831-bd4a-b976d10e964c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:18.109916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedfd5b-a360-4831-bd4a-b976d10e964c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:18.110705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedfd5b-a360-4831-bd4a-b976d10e964c None admin] RBAC: Adding query filter params (role.id=e3c5a858a45c41db8ff3e883788d75bc, scope.project.id=5445df38221b45d49ff15ad288d79d68, user.id=ec820f1dd7ee43e1ae477769a85fab85) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:18.111017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedfd5b-a360-4831-bd4a-b976d10e964c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:18.112475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedfd5b-a360-4831-bd4a-b976d10e964c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:18.118085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 565/1130] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:25:17 2022] GET /identity/v3/role_assignments?user.id=ec820f1dd7ee43e1ae477769a85fab85&scope.project.id=5445df38221b45d49ff15ad288d79d68&role.id=e3c5a858a45c41db8ff3e883788d75bc => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:20.105812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c2cf2-4245-453f-8fa0-b435b083af44 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:20.106348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c2cf2-4245-453f-8fa0-b435b083af44 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:20.106348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c2cf2-4245-453f-8fa0-b435b083af44 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:20.107694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 566/1131] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:20.112795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd2bbb-523b-41bf-bcd7-eb86834f7229 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:20.113183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd2bbb-523b-41bf-bcd7-eb86834f7229 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:20.113348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd2bbb-523b-41bf-bcd7-eb86834f7229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:20.131024 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-46fd2bbb-523b-41bf-bcd7-eb86834f7229 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:20.132215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-46fd2bbb-523b-41bf-bcd7-eb86834f7229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:20.195628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 566/1132] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:20.219421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c2cf2-4245-453f-8fa0-b435b083af44 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:20.285271 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:20.319507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c2cf2-4245-453f-8fa0-b435b083af44 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:20.353152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3f311-12a3-44df-835d-b7da07fd7b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:20.353780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3f311-12a3-44df-835d-b7da07fd7b43 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:20.354063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3f311-12a3-44df-835d-b7da07fd7b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:20.354339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3f311-12a3-44df-835d-b7da07fd7b43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:20.355734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3f311-12a3-44df-835d-b7da07fd7b43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:20.357198 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3f311-12a3-44df-835d-b7da07fd7b43 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:20.358850 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 567/1133] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:25:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:25:20.364671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd2bbb-523b-41bf-bcd7-eb86834f7229 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:20.372620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:20.462948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd2bbb-523b-41bf-bcd7-eb86834f7229 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:20.496369 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c03683-0667-4016-abf6-b6d5af30caa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:20.496916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03683-0667-4016-abf6-b6d5af30caa8 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:20.497202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03683-0667-4016-abf6-b6d5af30caa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:20.497479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03683-0667-4016-abf6-b6d5af30caa8 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:20.498164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c03683-0667-4016-abf6-b6d5af30caa8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:20.500174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c03683-0667-4016-abf6-b6d5af30caa8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:20.622135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 567/1134] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:25:20 2022] POST /identity/v3/projects => generated 311 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:25:22.649597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba00fa-27f7-4ab8-b222-a2946d95bde0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:22.650149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba00fa-27f7-4ab8-b222-a2946d95bde0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:22.650149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba00fa-27f7-4ab8-b222-a2946d95bde0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:22.651494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 568/1135] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:22.656679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4353a01-5be5-4cd0-b82a-21b05dde8a83 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:22.657072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4353a01-5be5-4cd0-b82a-21b05dde8a83 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:22.657229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4353a01-5be5-4cd0-b82a-21b05dde8a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:22.674835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f4353a01-5be5-4cd0-b82a-21b05dde8a83 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:22.676023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f4353a01-5be5-4cd0-b82a-21b05dde8a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:22.726539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:22.739811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 568/1136] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:22.765130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba00fa-27f7-4ab8-b222-a2946d95bde0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:22.865684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba00fa-27f7-4ab8-b222-a2946d95bde0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:22.898855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1a4bc-238d-4389-9c2d-99ee8a06cc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:22.899484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1a4bc-238d-4389-9c2d-99ee8a06cc93 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:22.899819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1a4bc-238d-4389-9c2d-99ee8a06cc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:22.900073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1a4bc-238d-4389-9c2d-99ee8a06cc93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:22.901213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1a4bc-238d-4389-9c2d-99ee8a06cc93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:22.902646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1a4bc-238d-4389-9c2d-99ee8a06cc93 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:22.904235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 569/1137] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:25:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:22.910005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4353a01-5be5-4cd0-b82a-21b05dde8a83 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:23.006519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4353a01-5be5-4cd0-b82a-21b05dde8a83 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:23.038585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e95935-c13f-41fe-9f08-57c4a37ebbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:23.039213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e95935-c13f-41fe-9f08-57c4a37ebbc7 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:23.039497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e95935-c13f-41fe-9f08-57c4a37ebbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:23.039771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e95935-c13f-41fe-9f08-57c4a37ebbc7 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:23.040509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e95935-c13f-41fe-9f08-57c4a37ebbc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:23.041937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e95935-c13f-41fe-9f08-57c4a37ebbc7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:23.119558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 569/1138] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:25:22 2022] POST /identity/v3/users => generated 286 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:25:25.139122 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f47791-b398-4fca-a0fd-0f9dc83229f1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:25.139669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f47791-b398-4fca-a0fd-0f9dc83229f1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:25.139669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f47791-b398-4fca-a0fd-0f9dc83229f1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:25.141102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 570/1139] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:25.146729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df4c95-6a03-46bf-bb71-67c6800eff03 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:25.147114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df4c95-6a03-46bf-bb71-67c6800eff03 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:25.147290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df4c95-6a03-46bf-bb71-67c6800eff03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:25.165140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-96df4c95-6a03-46bf-bb71-67c6800eff03 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:25.166325 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-96df4c95-6a03-46bf-bb71-67c6800eff03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:25.205622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:25.231148 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 570/1140] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:25.256057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f47791-b398-4fca-a0fd-0f9dc83229f1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:25.259768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:25.353691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f47791-b398-4fca-a0fd-0f9dc83229f1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:25.386621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9bf90-2bd6-466b-a649-22a4a05917ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:25.387242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9bf90-2bd6-466b-a649-22a4a05917ca None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:25.387521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9bf90-2bd6-466b-a649-22a4a05917ca None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:25.387791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9bf90-2bd6-466b-a649-22a4a05917ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:25.397669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9bf90-2bd6-466b-a649-22a4a05917ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bed9bf90-2bd6-466b-a649-22a4a05917ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:25:25.402467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:25.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:25:25.404837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 571/1141] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:25:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:25.409825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df4c95-6a03-46bf-bb71-67c6800eff03 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:25.506013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df4c95-6a03-46bf-bb71-67c6800eff03 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:25.538138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee6b6e-9400-4e65-8faf-b482ec33445e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:25.538746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee6b6e-9400-4e65-8faf-b482ec33445e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:25.539030 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee6b6e-9400-4e65-8faf-b482ec33445e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:25.539306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee6b6e-9400-4e65-8faf-b482ec33445e None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:25.540041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee6b6e-9400-4e65-8faf-b482ec33445e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:25.540350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee6b6e-9400-4e65-8faf-b482ec33445e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:25.541733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee6b6e-9400-4e65-8faf-b482ec33445e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:25.549972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 571/1142] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:25:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:25.555582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9bf90-2bd6-466b-a649-22a4a05917ca None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:25.617173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:25.653097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9bf90-2bd6-466b-a649-22a4a05917ca None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:25.686082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45477cdf-2f6d-4812-a0ce-78d4e87b816d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:25.686709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45477cdf-2f6d-4812-a0ce-78d4e87b816d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:25.686992 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45477cdf-2f6d-4812-a0ce-78d4e87b816d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:25.687270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45477cdf-2f6d-4812-a0ce-78d4e87b816d None admin] PATH_INFO: `/v3/users/22488ac19c3b4da1bea418220b97ccc7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:25.700057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45477cdf-2f6d-4812-a0ce-78d4e87b816d None admin] RBAC: Authorizing `identity:get_user(user_id=22488ac19c3b4da1bea418220b97ccc7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:25.701499 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45477cdf-2f6d-4812-a0ce-78d4e87b816d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:25.703097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 572/1143] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:25 2022] GET /identity/v3/users/22488ac19c3b4da1bea418220b97ccc7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:25.708952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee6b6e-9400-4e65-8faf-b482ec33445e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:25.743435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:25.804359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee6b6e-9400-4e65-8faf-b482ec33445e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:25.835701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8d8b7-1560-434b-a97b-b83d5fbe9cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:25.836327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d8b7-1560-434b-a97b-b83d5fbe9cdb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:25.836607 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d8b7-1560-434b-a97b-b83d5fbe9cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:25.836881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d8b7-1560-434b-a97b-b83d5fbe9cdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:25.900307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8d8b7-1560-434b-a97b-b83d5fbe9cdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:25.901737 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8d8b7-1560-434b-a97b-b83d5fbe9cdb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:25.933810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:25.996702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 572/1144] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:25:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:25:26.002597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45477cdf-2f6d-4812-a0ce-78d4e87b816d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:26.100477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45477cdf-2f6d-4812-a0ce-78d4e87b816d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:26.121842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:26.132781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb273d-d48e-43f1-baab-469ac870a624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:26.133409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb273d-d48e-43f1-baab-469ac870a624 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:26.133693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb273d-d48e-43f1-baab-469ac870a624 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:26.133970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb273d-d48e-43f1-baab-469ac870a624 None admin] PATH_INFO: `/v3/projects/b907e75117a94c8a86d17e20e904e942` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:26.135101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb273d-d48e-43f1-baab-469ac870a624 None admin] RBAC: Authorizing `identity:get_project(project_id=b907e75117a94c8a86d17e20e904e942)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:26.136603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb273d-d48e-43f1-baab-469ac870a624 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:26.141645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 573/1145] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:26 2022] GET /identity/v3/projects/b907e75117a94c8a86d17e20e904e942 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:26.145084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8d8b7-1560-434b-a97b-b83d5fbe9cdb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:26.238623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8d8b7-1560-434b-a97b-b83d5fbe9cdb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:26.270644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a5554-e8ea-42ae-bcd6-06b4a9258374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:26.271246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a5554-e8ea-42ae-bcd6-06b4a9258374 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:26.271527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a5554-e8ea-42ae-bcd6-06b4a9258374 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:26.271800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a5554-e8ea-42ae-bcd6-06b4a9258374 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:26.272768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a5554-e8ea-42ae-bcd6-06b4a9258374 None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=b907e75117a94c8a86d17e20e904e942, user.id=22488ac19c3b4da1bea418220b97ccc7) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:26.273077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a5554-e8ea-42ae-bcd6-06b4a9258374 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:26.274480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a5554-e8ea-42ae-bcd6-06b4a9258374 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:26.277177 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:26.279967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 573/1146] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:25:26 2022] GET /identity/v3/role_assignments?user.id=22488ac19c3b4da1bea418220b97ccc7&scope.project.id=b907e75117a94c8a86d17e20e904e942&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:28.257548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364ed19-60ee-469e-9737-bc1410df53b5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:28.258064 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364ed19-60ee-469e-9737-bc1410df53b5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:28.258064 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364ed19-60ee-469e-9737-bc1410df53b5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:28.259362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 574/1147] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:28.264571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a06b61-f3a8-4fbf-ad2d-211fef3d9098 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:28.264948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a06b61-f3a8-4fbf-ad2d-211fef3d9098 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:28.265136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a06b61-f3a8-4fbf-ad2d-211fef3d9098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:28.283805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e8a06b61-f3a8-4fbf-ad2d-211fef3d9098 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:28.285013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e8a06b61-f3a8-4fbf-ad2d-211fef3d9098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:28.348598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 574/1148] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:28.370577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364ed19-60ee-469e-9737-bc1410df53b5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:28.467901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364ed19-60ee-469e-9737-bc1410df53b5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:28.500879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21a70a-ed54-46d6-9763-29561b874088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:28.501496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21a70a-ed54-46d6-9763-29561b874088 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:28.501777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21a70a-ed54-46d6-9763-29561b874088 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:28.502051 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21a70a-ed54-46d6-9763-29561b874088 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:28.511889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21a70a-ed54-46d6-9763-29561b874088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ae21a70a-ed54-46d6-9763-29561b874088 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:25:28.517414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:28.518559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:25:28.519611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 575/1149] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:25:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:28.524852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a06b61-f3a8-4fbf-ad2d-211fef3d9098 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:28.619270 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a06b61-f3a8-4fbf-ad2d-211fef3d9098 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:28.646899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:28.652542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c0b40-14f5-457c-92e8-df80624c1d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:28.653147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c0b40-14f5-457c-92e8-df80624c1d91 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:28.653428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c0b40-14f5-457c-92e8-df80624c1d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:28.653703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c0b40-14f5-457c-92e8-df80624c1d91 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:28.654413 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c0b40-14f5-457c-92e8-df80624c1d91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:28.654718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c0b40-14f5-457c-92e8-df80624c1d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:28.656101 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c0b40-14f5-457c-92e8-df80624c1d91 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:28.664404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 575/1150] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:25:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:28.670384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21a70a-ed54-46d6-9763-29561b874088 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:28.719827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:28.768712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21a70a-ed54-46d6-9763-29561b874088 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:28.799975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa62414-9b36-46cb-a5a0-7c85bff16d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:28.800616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa62414-9b36-46cb-a5a0-7c85bff16d6c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:28.800893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa62414-9b36-46cb-a5a0-7c85bff16d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:28.801164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa62414-9b36-46cb-a5a0-7c85bff16d6c None admin] PATH_INFO: `/v3/users/22488ac19c3b4da1bea418220b97ccc7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:28.802290 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa62414-9b36-46cb-a5a0-7c85bff16d6c None admin] RBAC: Authorizing `identity:get_user(user_id=22488ac19c3b4da1bea418220b97ccc7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:28.803707 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa62414-9b36-46cb-a5a0-7c85bff16d6c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:28.805280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 576/1151] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:28 2022] GET /identity/v3/users/22488ac19c3b4da1bea418220b97ccc7 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:28.810963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c0b40-14f5-457c-92e8-df80624c1d91 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:28.907405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c0b40-14f5-457c-92e8-df80624c1d91 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:28.940573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355f1f6-fb49-4b82-b9a9-ca2796b1fced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:28.941200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355f1f6-fb49-4b82-b9a9-ca2796b1fced None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:28.941478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355f1f6-fb49-4b82-b9a9-ca2796b1fced None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:28.941750 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355f1f6-fb49-4b82-b9a9-ca2796b1fced None admin] PATH_INFO: `/v3/projects/b907e75117a94c8a86d17e20e904e942` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:28.942873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355f1f6-fb49-4b82-b9a9-ca2796b1fced None admin] RBAC: Authorizing `identity:get_project(project_id=b907e75117a94c8a86d17e20e904e942)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:28.944310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355f1f6-fb49-4b82-b9a9-ca2796b1fced None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:28.945915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 576/1152] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:28 2022] GET /identity/v3/projects/b907e75117a94c8a86d17e20e904e942 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:28.951820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa62414-9b36-46cb-a5a0-7c85bff16d6c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:29.049434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa62414-9b36-46cb-a5a0-7c85bff16d6c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:29.065960 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:29.081749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea24a7e-b49d-48a3-8d08-f246300023b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:29.084780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea24a7e-b49d-48a3-8d08-f246300023b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:29.084780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea24a7e-b49d-48a3-8d08-f246300023b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:29.084780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea24a7e-b49d-48a3-8d08-f246300023b3 None admin] PATH_INFO: `/v3/projects/b907e75117a94c8a86d17e20e904e942/users/22488ac19c3b4da1bea418220b97ccc7/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:29.085040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea24a7e-b49d-48a3-8d08-f246300023b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b907e75117a94c8a86d17e20e904e942, user_id=22488ac19c3b4da1bea418220b97ccc7, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:29.086231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea24a7e-b49d-48a3-8d08-f246300023b3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:29.170616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 577/1153] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:25:28 2022] PUT /identity/v3/projects/b907e75117a94c8a86d17e20e904e942/users/22488ac19c3b4da1bea418220b97ccc7/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:25:31.133733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1d05-1732-4211-9c8e-b259d35970cc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:31.134277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1d05-1732-4211-9c8e-b259d35970cc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:31.134277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc1d05-1732-4211-9c8e-b259d35970cc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:31.135604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 577/1154] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:31.140740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe481b-21eb-4f7c-9d19-c03132b61ac4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:31.141150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe481b-21eb-4f7c-9d19-c03132b61ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:31.141391 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe481b-21eb-4f7c-9d19-c03132b61ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:31.159609 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dafe481b-21eb-4f7c-9d19-c03132b61ac4 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:31.160794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dafe481b-21eb-4f7c-9d19-c03132b61ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:31.224747 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 578/1155] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:31.247400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc1d05-1732-4211-9c8e-b259d35970cc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:31.305353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:31.345748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc1d05-1732-4211-9c8e-b259d35970cc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:31.378479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf11f10-c859-457a-9b49-f7ea9e705acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:31.379106 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf11f10-c859-457a-9b49-f7ea9e705acd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:31.379392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf11f10-c859-457a-9b49-f7ea9e705acd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:31.379672 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf11f10-c859-457a-9b49-f7ea9e705acd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:31.390479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf11f10-c859-457a-9b49-f7ea9e705acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0cf11f10-c859-457a-9b49-f7ea9e705acd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:25:31.395922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:31.397129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:25:31.398253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 578/1156] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:25:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:31.402940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe481b-21eb-4f7c-9d19-c03132b61ac4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:31.497510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe481b-21eb-4f7c-9d19-c03132b61ac4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:31.506261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:31.530054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0137a2-409d-459a-bace-cc0ecf52a731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:31.530653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0137a2-409d-459a-bace-cc0ecf52a731 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:31.530935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0137a2-409d-459a-bace-cc0ecf52a731 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:31.531210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0137a2-409d-459a-bace-cc0ecf52a731 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:31.532553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0137a2-409d-459a-bace-cc0ecf52a731 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:31.532856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0137a2-409d-459a-bace-cc0ecf52a731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:31.534278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0137a2-409d-459a-bace-cc0ecf52a731 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:31.542325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 579/1157] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:25:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:31.548143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf11f10-c859-457a-9b49-f7ea9e705acd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:31.647008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf11f10-c859-457a-9b49-f7ea9e705acd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:31.665554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:31.678784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79981cac-344f-4cb6-8070-1ca1a043fa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:31.679413 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79981cac-344f-4cb6-8070-1ca1a043fa67 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:31.679696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79981cac-344f-4cb6-8070-1ca1a043fa67 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:31.679976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79981cac-344f-4cb6-8070-1ca1a043fa67 None admin] PATH_INFO: `/v3/users/22488ac19c3b4da1bea418220b97ccc7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:31.681141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79981cac-344f-4cb6-8070-1ca1a043fa67 None admin] RBAC: Authorizing `identity:get_user(user_id=22488ac19c3b4da1bea418220b97ccc7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:31.682616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79981cac-344f-4cb6-8070-1ca1a043fa67 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:31.684214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 579/1158] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:31 2022] GET /identity/v3/users/22488ac19c3b4da1bea418220b97ccc7 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:31.690591 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0137a2-409d-459a-bace-cc0ecf52a731 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:31.787273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0137a2-409d-459a-bace-cc0ecf52a731 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:31.819682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787968f-ec3a-4c2c-8669-45a5638080ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:31.820308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787968f-ec3a-4c2c-8669-45a5638080ed None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:31.820593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787968f-ec3a-4c2c-8669-45a5638080ed None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:31.820869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787968f-ec3a-4c2c-8669-45a5638080ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:31.858489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:31.886011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787968f-ec3a-4c2c-8669-45a5638080ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:31.887461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787968f-ec3a-4c2c-8669-45a5638080ed None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:31.982623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 580/1159] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:25:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:25:31.989348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79981cac-344f-4cb6-8070-1ca1a043fa67 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:32.084794 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79981cac-344f-4cb6-8070-1ca1a043fa67 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:32.118503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328a965-f8e6-4ea7-8d87-0d180645977f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:32.119129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328a965-f8e6-4ea7-8d87-0d180645977f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:32.119409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328a965-f8e6-4ea7-8d87-0d180645977f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:32.119681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328a965-f8e6-4ea7-8d87-0d180645977f None admin] PATH_INFO: `/v3/projects/b907e75117a94c8a86d17e20e904e942` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:32.120832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328a965-f8e6-4ea7-8d87-0d180645977f None admin] RBAC: Authorizing `identity:get_project(project_id=b907e75117a94c8a86d17e20e904e942)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:32.122245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328a965-f8e6-4ea7-8d87-0d180645977f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:32.123850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 580/1160] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:31 2022] GET /identity/v3/projects/b907e75117a94c8a86d17e20e904e942 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:32.130553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787968f-ec3a-4c2c-8669-45a5638080ed None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:32.196180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:32.225679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787968f-ec3a-4c2c-8669-45a5638080ed None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:32.257639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52410c3-f158-4b2d-a6c6-4ee3abf7e274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:32.258228 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52410c3-f158-4b2d-a6c6-4ee3abf7e274 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:32.258504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52410c3-f158-4b2d-a6c6-4ee3abf7e274 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:32.258774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52410c3-f158-4b2d-a6c6-4ee3abf7e274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:32.259533 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52410c3-f158-4b2d-a6c6-4ee3abf7e274 None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=b907e75117a94c8a86d17e20e904e942, user.id=22488ac19c3b4da1bea418220b97ccc7) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:32.259841 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52410c3-f158-4b2d-a6c6-4ee3abf7e274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:32.261267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52410c3-f158-4b2d-a6c6-4ee3abf7e274 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:32.267323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 581/1161] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:25:32 2022] GET /identity/v3/role_assignments?user.id=22488ac19c3b4da1bea418220b97ccc7&scope.project.id=b907e75117a94c8a86d17e20e904e942&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:34.280734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c43ff-5898-4d8f-8245-802fa8170e67 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:34.281302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c43ff-5898-4d8f-8245-802fa8170e67 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:34.281302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c43ff-5898-4d8f-8245-802fa8170e67 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:34.283247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 581/1162] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:34.288472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716e136-c879-4070-903c-ea5cdae6e2cf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:34.288845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716e136-c879-4070-903c-ea5cdae6e2cf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:34.289035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716e136-c879-4070-903c-ea5cdae6e2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:34.306664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1716e136-c879-4070-903c-ea5cdae6e2cf None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:34.307838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1716e136-c879-4070-903c-ea5cdae6e2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:34.372329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 582/1163] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:34.394719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c43ff-5898-4d8f-8245-802fa8170e67 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:34.449808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:34.493651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c43ff-5898-4d8f-8245-802fa8170e67 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:34.525464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2f757-eccd-4cf8-b97b-c7a264534fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:34.526058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2f757-eccd-4cf8-b97b-c7a264534fed None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:34.526337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2f757-eccd-4cf8-b97b-c7a264534fed None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:34.526609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2f757-eccd-4cf8-b97b-c7a264534fed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:34.590558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2f757-eccd-4cf8-b97b-c7a264534fed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:34.591918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2f757-eccd-4cf8-b97b-c7a264534fed None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:34.643886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:34.685780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 582/1164] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:25:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:25:34.692345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716e136-c879-4070-903c-ea5cdae6e2cf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:34.752978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:34.789200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716e136-c879-4070-903c-ea5cdae6e2cf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:34.821714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524b130-20aa-431d-8650-3ad5c6061351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:34.822340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524b130-20aa-431d-8650-3ad5c6061351 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:34.822623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524b130-20aa-431d-8650-3ad5c6061351 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:34.822898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524b130-20aa-431d-8650-3ad5c6061351 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:34.829903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524b130-20aa-431d-8650-3ad5c6061351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:34.831343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524b130-20aa-431d-8650-3ad5c6061351 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9524b130-20aa-431d-8650-3ad5c6061351 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:34.842538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:34.843752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:25:34.844823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 583/1165] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Aug 4 07:25:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:34.849633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2f757-eccd-4cf8-b97b-c7a264534fed None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:34.944479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2f757-eccd-4cf8-b97b-c7a264534fed None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:34.979272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccf2d3-c28b-46ba-b7ec-ad000c4ce2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:34.980161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccf2d3-c28b-46ba-b7ec-ad000c4ce2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:34.980444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccf2d3-c28b-46ba-b7ec-ad000c4ce2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:34.980718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccf2d3-c28b-46ba-b7ec-ad000c4ce2e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:34.981398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccf2d3-c28b-46ba-b7ec-ad000c4ce2e4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:34.981701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccf2d3-c28b-46ba-b7ec-ad000c4ce2e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:34.983056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccf2d3-c28b-46ba-b7ec-ad000c4ce2e4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:34.990783 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 583/1166] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Aug 4 07:25:34 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:36.978248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a080e-136a-4129-b092-c42d2c147d60 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:36.978684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a080e-136a-4129-b092-c42d2c147d60 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:36.978790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a080e-136a-4129-b092-c42d2c147d60 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:36.980427 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 584/1167] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:36.986310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17605cca-cb7e-41aa-a32c-b2131e7102bf None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:36.986694 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17605cca-cb7e-41aa-a32c-b2131e7102bf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:36.986878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17605cca-cb7e-41aa-a32c-b2131e7102bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:37.005365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-17605cca-cb7e-41aa-a32c-b2131e7102bf None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:37.006557 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-17605cca-cb7e-41aa-a32c-b2131e7102bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:37.020218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:37.072946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 584/1168] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:37.096076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a080e-136a-4129-b092-c42d2c147d60 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:37.194265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a080e-136a-4129-b092-c42d2c147d60 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:37.216835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:37.227471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050a83-6938-405c-a5cd-d72b5c86b60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:37.228101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050a83-6938-405c-a5cd-d72b5c86b60a None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:37.228387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050a83-6938-405c-a5cd-d72b5c86b60a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:37.228672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050a83-6938-405c-a5cd-d72b5c86b60a None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:37.229382 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e050a83-6938-405c-a5cd-d72b5c86b60a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:37.231013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e050a83-6938-405c-a5cd-d72b5c86b60a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:37.324413 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 585/1169] 10.222.0.64 () {60 vars in 1138 bytes} [Thu Aug 4 07:25:37 2022] POST /identity/v3/domains => generated 267 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:25:39.323352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d811dcb-c7ae-461c-bf7c-14bfc87f39f5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:39.323950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d811dcb-c7ae-461c-bf7c-14bfc87f39f5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:39.323950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d811dcb-c7ae-461c-bf7c-14bfc87f39f5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:39.325235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 585/1170] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:39.331008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485db00-60a4-4f04-812c-00689530ab32 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:39.331407 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485db00-60a4-4f04-812c-00689530ab32 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:39.331561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485db00-60a4-4f04-812c-00689530ab32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:39.348998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2485db00-60a4-4f04-812c-00689530ab32 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:39.350158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2485db00-60a4-4f04-812c-00689530ab32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:39.413182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 586/1171] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:39.436769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d811dcb-c7ae-461c-bf7c-14bfc87f39f5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:39.533564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d811dcb-c7ae-461c-bf7c-14bfc87f39f5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:39.566325 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ae038-a690-47ee-a0b7-ab6cf62d8f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:39.566942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ae038-a690-47ee-a0b7-ab6cf62d8f58 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:39.567225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ae038-a690-47ee-a0b7-ab6cf62d8f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:39.567503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ae038-a690-47ee-a0b7-ab6cf62d8f58 None admin] PATH_INFO: `/v3/domains/02ba1131eb134aebb8cd53ed2869113d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:39.577495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ae038-a690-47ee-a0b7-ab6cf62d8f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=02ba1131eb134aebb8cd53ed2869113d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:39.578911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ae038-a690-47ee-a0b7-ab6cf62d8f58 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:39.580505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 586/1172] 10.222.0.64 () {56 vars in 1183 bytes} [Thu Aug 4 07:25:39 2022] GET /identity/v3/domains/02ba1131eb134aebb8cd53ed2869113d => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:39.586293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485db00-60a4-4f04-812c-00689530ab32 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:39.679374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485db00-60a4-4f04-812c-00689530ab32 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:39.692884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:39.711416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac950b0-7321-4ec4-bb01-fbad9afed004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:39.712032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac950b0-7321-4ec4-bb01-fbad9afed004 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:39.712313 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac950b0-7321-4ec4-bb01-fbad9afed004 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:39.712585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac950b0-7321-4ec4-bb01-fbad9afed004 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:39.713246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac950b0-7321-4ec4-bb01-fbad9afed004 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:39.714666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac950b0-7321-4ec4-bb01-fbad9afed004 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:39.829228 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 587/1173] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:25:39 2022] POST /identity/v3/projects => generated 361 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:25:42.071148 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6d6d5-df9c-4a4e-9f53-ffffe702c5f3 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:42.071666 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6d6d5-df9c-4a4e-9f53-ffffe702c5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:42.071757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6d6d5-df9c-4a4e-9f53-ffffe702c5f3 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:42.073076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 587/1174] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:42.079082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bf16f-a243-4892-969e-158cb57b4751 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:42.079512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bf16f-a243-4892-969e-158cb57b4751 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:42.079659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bf16f-a243-4892-969e-158cb57b4751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:42.097443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3f8bf16f-a243-4892-969e-158cb57b4751 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:42.098607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3f8bf16f-a243-4892-969e-158cb57b4751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:42.163000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 588/1175] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:42.188253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6d6d5-df9c-4a4e-9f53-ffffe702c5f3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:42.200799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:42.286366 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6d6d5-df9c-4a4e-9f53-ffffe702c5f3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:42.318228 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f787242-7233-4201-a6d8-d9585bd93109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:42.318850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f787242-7233-4201-a6d8-d9585bd93109 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:42.319132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f787242-7233-4201-a6d8-d9585bd93109 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:42.319413 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f787242-7233-4201-a6d8-d9585bd93109 None admin] PATH_INFO: `/v3/domains/02ba1131eb134aebb8cd53ed2869113d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:42.320572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f787242-7233-4201-a6d8-d9585bd93109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=02ba1131eb134aebb8cd53ed2869113d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:42.321989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f787242-7233-4201-a6d8-d9585bd93109 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:42.323564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 588/1176] 10.222.0.64 () {56 vars in 1183 bytes} [Thu Aug 4 07:25:42 2022] GET /identity/v3/domains/02ba1131eb134aebb8cd53ed2869113d => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:42.330053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bf16f-a243-4892-969e-158cb57b4751 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:42.417997 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:42.423763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bf16f-a243-4892-969e-158cb57b4751 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:42.456039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56b49b-719e-446e-8470-e3a92742c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:42.456990 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56b49b-719e-446e-8470-e3a92742c4d4 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:42.457445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56b49b-719e-446e-8470-e3a92742c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:42.458094 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56b49b-719e-446e-8470-e3a92742c4d4 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:42.458823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56b49b-719e-446e-8470-e3a92742c4d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:42.460278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56b49b-719e-446e-8470-e3a92742c4d4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:42.552180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 589/1177] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:25:42 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:25:44.570193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a42a-829e-4e45-92ed-519a75e3c801 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:44.570720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a42a-829e-4e45-92ed-519a75e3c801 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:44.570720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a42a-829e-4e45-92ed-519a75e3c801 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:44.572223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 589/1178] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:44.577315 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e3868-5a53-4d13-a4e5-26386403a14d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:44.578307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e3868-5a53-4d13-a4e5-26386403a14d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:44.578581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e3868-5a53-4d13-a4e5-26386403a14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:44.596296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f70e3868-5a53-4d13-a4e5-26386403a14d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:44.597457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f70e3868-5a53-4d13-a4e5-26386403a14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:44.660840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 590/1179] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:44.684727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488a42a-829e-4e45-92ed-519a75e3c801 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:44.761496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:44.782671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488a42a-829e-4e45-92ed-519a75e3c801 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:44.814937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4c2eb-013b-4971-b9d6-6a930ed5b130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:44.815593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4c2eb-013b-4971-b9d6-6a930ed5b130 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:44.816268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4c2eb-013b-4971-b9d6-6a930ed5b130 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:44.816548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4c2eb-013b-4971-b9d6-6a930ed5b130 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:44.826697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4c2eb-013b-4971-b9d6-6a930ed5b130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-83b4c2eb-013b-4971-b9d6-6a930ed5b130 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:25:44.831493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:44.832651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:25:44.833665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 590/1180] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:25:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:25:44.838716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e3868-5a53-4d13-a4e5-26386403a14d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:44.915903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:44.933459 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e3868-5a53-4d13-a4e5-26386403a14d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:44.966642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d77e43-6aba-4894-8243-11221f6fb0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:44.967278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d77e43-6aba-4894-8243-11221f6fb0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:44.967556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d77e43-6aba-4894-8243-11221f6fb0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:44.967829 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d77e43-6aba-4894-8243-11221f6fb0f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:44.968791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d77e43-6aba-4894-8243-11221f6fb0f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:44.969095 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d77e43-6aba-4894-8243-11221f6fb0f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:44.970452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d77e43-6aba-4894-8243-11221f6fb0f0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:44.978806 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 591/1181] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:25:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:44.985534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4c2eb-013b-4971-b9d6-6a930ed5b130 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:45.081218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4c2eb-013b-4971-b9d6-6a930ed5b130 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:45.114573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62b27d-7707-4449-b7ac-35c1f1691c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:45.115203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62b27d-7707-4449-b7ac-35c1f1691c30 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:45.115488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62b27d-7707-4449-b7ac-35c1f1691c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:45.115765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62b27d-7707-4449-b7ac-35c1f1691c30 None admin] PATH_INFO: `/v3/users/45acf3ba876d4a7c8ad44542f935a29d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:45.123243 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:45.128733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62b27d-7707-4449-b7ac-35c1f1691c30 None admin] RBAC: Authorizing `identity:get_user(user_id=45acf3ba876d4a7c8ad44542f935a29d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:45.130205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62b27d-7707-4449-b7ac-35c1f1691c30 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:45.131806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 591/1182] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:44 2022] GET /identity/v3/users/45acf3ba876d4a7c8ad44542f935a29d => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:45.137642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d77e43-6aba-4894-8243-11221f6fb0f0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:45.234498 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d77e43-6aba-4894-8243-11221f6fb0f0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:45.266684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4f7af-b83c-4196-b7f7-c75cdc22c7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:45.267285 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4f7af-b83c-4196-b7f7-c75cdc22c7fe None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:45.267567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4f7af-b83c-4196-b7f7-c75cdc22c7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:45.267839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4f7af-b83c-4196-b7f7-c75cdc22c7fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:45.282135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:45.333654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4f7af-b83c-4196-b7f7-c75cdc22c7fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:45.335176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4f7af-b83c-4196-b7f7-c75cdc22c7fe None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:45.434429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 592/1183] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:25:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:25:45.440535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62b27d-7707-4449-b7ac-35c1f1691c30 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:45.536049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62b27d-7707-4449-b7ac-35c1f1691c30 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:45.567748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a373833-a891-47d5-a6b7-54b5c1dd6659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:45.568410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a373833-a891-47d5-a6b7-54b5c1dd6659 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:45.568696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a373833-a891-47d5-a6b7-54b5c1dd6659 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:45.568974 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a373833-a891-47d5-a6b7-54b5c1dd6659 None admin] PATH_INFO: `/v3/projects/de2eb522d37b4a2d82994e5dfb2ab363` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:45.570141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a373833-a891-47d5-a6b7-54b5c1dd6659 None admin] RBAC: Authorizing `identity:get_project(project_id=de2eb522d37b4a2d82994e5dfb2ab363)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:45.571580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a373833-a891-47d5-a6b7-54b5c1dd6659 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:45.573264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 592/1184] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:45 2022] GET /identity/v3/projects/de2eb522d37b4a2d82994e5dfb2ab363 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:45.579848 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4f7af-b83c-4196-b7f7-c75cdc22c7fe None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:45.623595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:45.676964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4f7af-b83c-4196-b7f7-c75cdc22c7fe None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:45.710572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5208ba8-d75e-40dc-b463-dc916ab5d9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:45.711171 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5208ba8-d75e-40dc-b463-dc916ab5d9df None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:45.711457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5208ba8-d75e-40dc-b463-dc916ab5d9df None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:45.711811 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5208ba8-d75e-40dc-b463-dc916ab5d9df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:45.712946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5208ba8-d75e-40dc-b463-dc916ab5d9df None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=de2eb522d37b4a2d82994e5dfb2ab363, user.id=45acf3ba876d4a7c8ad44542f935a29d) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:45.713415 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5208ba8-d75e-40dc-b463-dc916ab5d9df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:45.715469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5208ba8-d75e-40dc-b463-dc916ab5d9df None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:45.722338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 593/1185] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:25:45 2022] GET /identity/v3/role_assignments?user.id=45acf3ba876d4a7c8ad44542f935a29d&scope.project.id=de2eb522d37b4a2d82994e5dfb2ab363&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:47.709428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7501fe-4279-4e2e-9225-a817c2e8455d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:47.710034 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7501fe-4279-4e2e-9225-a817c2e8455d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:47.710034 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7501fe-4279-4e2e-9225-a817c2e8455d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:47.711317 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 593/1186] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:47.716697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5feebf-10f6-4cf4-b0bf-f142fe127491 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:47.724810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5feebf-10f6-4cf4-b0bf-f142fe127491 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:47.724810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5feebf-10f6-4cf4-b0bf-f142fe127491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:47.735688 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cd5feebf-10f6-4cf4-b0bf-f142fe127491 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:47.736930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cd5feebf-10f6-4cf4-b0bf-f142fe127491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:47.800604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 594/1187] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:47.822772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7501fe-4279-4e2e-9225-a817c2e8455d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:47.883996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:47.920427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7501fe-4279-4e2e-9225-a817c2e8455d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:47.953448 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07e2de-d16a-46a3-aa7b-78d2ad9333b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:47.954078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07e2de-d16a-46a3-aa7b-78d2ad9333b1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:47.954358 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07e2de-d16a-46a3-aa7b-78d2ad9333b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:47.954632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07e2de-d16a-46a3-aa7b-78d2ad9333b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:47.964648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07e2de-d16a-46a3-aa7b-78d2ad9333b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ce07e2de-d16a-46a3-aa7b-78d2ad9333b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:25:47.969579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:47.970793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:25:47.972075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 594/1188] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:25:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:47.976778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5feebf-10f6-4cf4-b0bf-f142fe127491 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:48.015643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:48.073088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5feebf-10f6-4cf4-b0bf-f142fe127491 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:48.106194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb05692-7939-4afd-a9cf-d4565a70c652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:48.106798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb05692-7939-4afd-a9cf-d4565a70c652 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:48.107082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb05692-7939-4afd-a9cf-d4565a70c652 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:48.107360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb05692-7939-4afd-a9cf-d4565a70c652 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:48.108091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb05692-7939-4afd-a9cf-d4565a70c652 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:48.108398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb05692-7939-4afd-a9cf-d4565a70c652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:48.110357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb05692-7939-4afd-a9cf-d4565a70c652 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:48.118099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 595/1189] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:25:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:48.124028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07e2de-d16a-46a3-aa7b-78d2ad9333b1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:48.220309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07e2de-d16a-46a3-aa7b-78d2ad9333b1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:48.244346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:48.254649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4135a1-3974-44bf-a27e-d18d48de3635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:48.255276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4135a1-3974-44bf-a27e-d18d48de3635 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:48.255562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4135a1-3974-44bf-a27e-d18d48de3635 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:48.255842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4135a1-3974-44bf-a27e-d18d48de3635 None admin] PATH_INFO: `/v3/users/45acf3ba876d4a7c8ad44542f935a29d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:48.257024 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4135a1-3974-44bf-a27e-d18d48de3635 None admin] RBAC: Authorizing `identity:get_user(user_id=45acf3ba876d4a7c8ad44542f935a29d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:48.258450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4135a1-3974-44bf-a27e-d18d48de3635 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:48.260161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 595/1190] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:48 2022] GET /identity/v3/users/45acf3ba876d4a7c8ad44542f935a29d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:48.265942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb05692-7939-4afd-a9cf-d4565a70c652 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:48.361660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:48.364556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb05692-7939-4afd-a9cf-d4565a70c652 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:48.397743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd6e6a-1368-4818-8e69-827b6d7b6dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:48.398371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd6e6a-1368-4818-8e69-827b6d7b6dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:48.398654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd6e6a-1368-4818-8e69-827b6d7b6dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:48.398933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd6e6a-1368-4818-8e69-827b6d7b6dc9 None admin] PATH_INFO: `/v3/projects/de2eb522d37b4a2d82994e5dfb2ab363` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:48.400098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd6e6a-1368-4818-8e69-827b6d7b6dc9 None admin] RBAC: Authorizing `identity:get_project(project_id=de2eb522d37b4a2d82994e5dfb2ab363)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:48.401528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd6e6a-1368-4818-8e69-827b6d7b6dc9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:48.403159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 596/1191] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:48 2022] GET /identity/v3/projects/de2eb522d37b4a2d82994e5dfb2ab363 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:48.409603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4135a1-3974-44bf-a27e-d18d48de3635 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:48.507158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4135a1-3974-44bf-a27e-d18d48de3635 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:48.540592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399bc69-9c59-437d-9c43-bb8ba22d0b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:48.541246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399bc69-9c59-437d-9c43-bb8ba22d0b42 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:48.541525 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399bc69-9c59-437d-9c43-bb8ba22d0b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:48.541800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399bc69-9c59-437d-9c43-bb8ba22d0b42 None admin] PATH_INFO: `/v3/projects/de2eb522d37b4a2d82994e5dfb2ab363/users/45acf3ba876d4a7c8ad44542f935a29d/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:48.543705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399bc69-9c59-437d-9c43-bb8ba22d0b42 None admin] RBAC: Authorizing `identity:create_grant(project_id=de2eb522d37b4a2d82994e5dfb2ab363, user_id=45acf3ba876d4a7c8ad44542f935a29d, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:48.545158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399bc69-9c59-437d-9c43-bb8ba22d0b42 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:48.614255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 596/1192] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:25:48 2022] PUT /identity/v3/projects/de2eb522d37b4a2d82994e5dfb2ab363/users/45acf3ba876d4a7c8ad44542f935a29d/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:25:50.573584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876e827-6711-4291-a60a-20cd5cf5e73a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:50.574134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876e827-6711-4291-a60a-20cd5cf5e73a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:50.574134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876e827-6711-4291-a60a-20cd5cf5e73a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:50.575456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 597/1193] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:25:50.581180 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ae41b-adc7-481d-a03d-a07a4c2aa809 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:50.581609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ae41b-adc7-481d-a03d-a07a4c2aa809 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:50.581740 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ae41b-adc7-481d-a03d-a07a4c2aa809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:50.599286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a65ae41b-adc7-481d-a03d-a07a4c2aa809 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:25:50.600502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a65ae41b-adc7-481d-a03d-a07a4c2aa809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:25:50.646093 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:50.665407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 597/1194] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:25:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:25:50.689713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876e827-6711-4291-a60a-20cd5cf5e73a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:50.788530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876e827-6711-4291-a60a-20cd5cf5e73a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:50.820569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf359c-7e34-4c35-8003-c19e5486f14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:50.821192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf359c-7e34-4c35-8003-c19e5486f14e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:50.821475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf359c-7e34-4c35-8003-c19e5486f14e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:50.821750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf359c-7e34-4c35-8003-c19e5486f14e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:50.832131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf359c-7e34-4c35-8003-c19e5486f14e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e9bf359c-7e34-4c35-8003-c19e5486f14e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:25:50.836916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:25:50.837988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:25:50.839077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:25:50.840133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 598/1195] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:25:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:25:50.844847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ae41b-adc7-481d-a03d-a07a4c2aa809 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:50.939392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ae41b-adc7-481d-a03d-a07a4c2aa809 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:50.973935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ed95f-3d19-4b91-b0c1-0cb564862c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:50.974552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ed95f-3d19-4b91-b0c1-0cb564862c8a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:50.974837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ed95f-3d19-4b91-b0c1-0cb564862c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:50.975112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ed95f-3d19-4b91-b0c1-0cb564862c8a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:50.975825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ed95f-3d19-4b91-b0c1-0cb564862c8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:50.976166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ed95f-3d19-4b91-b0c1-0cb564862c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:50.977537 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ed95f-3d19-4b91-b0c1-0cb564862c8a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:50.985747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 598/1196] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:25:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:25:50.991493 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf359c-7e34-4c35-8003-c19e5486f14e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:51.019185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:51.087376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf359c-7e34-4c35-8003-c19e5486f14e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:51.118964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f973cbc-57ef-41e9-9d30-aa7dbe786d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:51.119577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f973cbc-57ef-41e9-9d30-aa7dbe786d2a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:51.119854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f973cbc-57ef-41e9-9d30-aa7dbe786d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:51.120150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f973cbc-57ef-41e9-9d30-aa7dbe786d2a None admin] PATH_INFO: `/v3/users/45acf3ba876d4a7c8ad44542f935a29d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:51.121270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f973cbc-57ef-41e9-9d30-aa7dbe786d2a None admin] RBAC: Authorizing `identity:get_user(user_id=45acf3ba876d4a7c8ad44542f935a29d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:51.122684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f973cbc-57ef-41e9-9d30-aa7dbe786d2a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:51.124787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 599/1197] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Aug 4 07:25:50 2022] GET /identity/v3/users/45acf3ba876d4a7c8ad44542f935a29d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:51.130544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ed95f-3d19-4b91-b0c1-0cb564862c8a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:51.174638 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:51.229960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ed95f-3d19-4b91-b0c1-0cb564862c8a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:51.262395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee1e02-51fa-4ac8-87b5-4b109a5e9bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:51.262997 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee1e02-51fa-4ac8-87b5-4b109a5e9bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:51.263277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee1e02-51fa-4ac8-87b5-4b109a5e9bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:51.263548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee1e02-51fa-4ac8-87b5-4b109a5e9bd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:51.328250 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee1e02-51fa-4ac8-87b5-4b109a5e9bd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:51.329674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee1e02-51fa-4ac8-87b5-4b109a5e9bd0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:51.370565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:51.423050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 599/1198] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:25:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:25:51.428906 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f973cbc-57ef-41e9-9d30-aa7dbe786d2a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:51.519416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:25:51.524916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f973cbc-57ef-41e9-9d30-aa7dbe786d2a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:51.558001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227786a7-5646-4ac2-a331-efacf8983965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:51.558621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227786a7-5646-4ac2-a331-efacf8983965 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:51.558899 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227786a7-5646-4ac2-a331-efacf8983965 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:51.559171 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227786a7-5646-4ac2-a331-efacf8983965 None admin] PATH_INFO: `/v3/projects/de2eb522d37b4a2d82994e5dfb2ab363` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:51.560312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227786a7-5646-4ac2-a331-efacf8983965 None admin] RBAC: Authorizing `identity:get_project(project_id=de2eb522d37b4a2d82994e5dfb2ab363)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:51.562432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227786a7-5646-4ac2-a331-efacf8983965 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:51.564064 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 600/1199] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Aug 4 07:25:51 2022] GET /identity/v3/projects/de2eb522d37b4a2d82994e5dfb2ab363 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:25:51.570086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee1e02-51fa-4ac8-87b5-4b109a5e9bd0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:25:51.663424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee1e02-51fa-4ac8-87b5-4b109a5e9bd0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:25:51.697408 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aecf7c-3948-450a-9ac9-6bd72381c224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:25:51.698015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aecf7c-3948-450a-9ac9-6bd72381c224 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:25:51.698301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aecf7c-3948-450a-9ac9-6bd72381c224 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:25:51.698580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aecf7c-3948-450a-9ac9-6bd72381c224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:25:51.699368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aecf7c-3948-450a-9ac9-6bd72381c224 None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=de2eb522d37b4a2d82994e5dfb2ab363, user.id=45acf3ba876d4a7c8ad44542f935a29d) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:25:51.699681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aecf7c-3948-450a-9ac9-6bd72381c224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:25:51.701142 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aecf7c-3948-450a-9ac9-6bd72381c224 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:25:51.707093 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 600/1200] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:25:51 2022] GET /identity/v3/role_assignments?user.id=45acf3ba876d4a7c8ad44542f935a29d&scope.project.id=de2eb522d37b4a2d82994e5dfb2ab363&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:26:01.553952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=18 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:26:01.704535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:03.824886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c357270-5c5b-45da-9b1e-c023d806c25b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:03.825531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c357270-5c5b-45da-9b1e-c023d806c25b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:03.825531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c357270-5c5b-45da-9b1e-c023d806c25b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:03.827393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 601/1201] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:27:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:27:03.832592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3ac9c-7bdb-443d-a193-f6975d9ede40 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:03.833033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3ac9c-7bdb-443d-a193-f6975d9ede40 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:03.833136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3ac9c-7bdb-443d-a193-f6975d9ede40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:03.851511 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-36e3ac9c-7bdb-443d-a193-f6975d9ede40 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:27:03.852991 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-36e3ac9c-7bdb-443d-a193-f6975d9ede40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:27:03.917220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 601/1202] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:27:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:27:03.942160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c357270-5c5b-45da-9b1e-c023d806c25b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:04.043240 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c357270-5c5b-45da-9b1e-c023d806c25b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:04.077790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eca31f-61a0-450f-9498-706cb20f5f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:04.078340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eca31f-61a0-450f-9498-706cb20f5f21 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:04.078621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eca31f-61a0-450f-9498-706cb20f5f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:04.078893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eca31f-61a0-450f-9498-706cb20f5f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:04.089019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eca31f-61a0-450f-9498-706cb20f5f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:04.093437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eca31f-61a0-450f-9498-706cb20f5f21 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-35eca31f-61a0-450f-9498-706cb20f5f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:04.105601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:04.107456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:04.108691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:04.108691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:04.108691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:04.108691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:04.108691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 602/1203] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:04.113491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3ac9c-7bdb-443d-a193-f6975d9ede40 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:04.210627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3ac9c-7bdb-443d-a193-f6975d9ede40 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:04.238558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:04.244141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14c6ce-2f27-49ce-bb7a-a8c1c6b965f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:04.244749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14c6ce-2f27-49ce-bb7a-a8c1c6b965f4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:04.245030 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14c6ce-2f27-49ce-bb7a-a8c1c6b965f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:04.245302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14c6ce-2f27-49ce-bb7a-a8c1c6b965f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:04.245981 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14c6ce-2f27-49ce-bb7a-a8c1c6b965f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:04.246283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14c6ce-2f27-49ce-bb7a-a8c1c6b965f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:04.248276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14c6ce-2f27-49ce-bb7a-a8c1c6b965f4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:04.258991 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 602/1204] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:04.264869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eca31f-61a0-450f-9498-706cb20f5f21 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:04.317607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:04.364541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eca31f-61a0-450f-9498-706cb20f5f21 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:04.398666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69004ac6-9122-4ad9-985c-9d337baacd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:04.399271 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69004ac6-9122-4ad9-985c-9d337baacd5e None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:04.399555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69004ac6-9122-4ad9-985c-9d337baacd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:04.399829 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69004ac6-9122-4ad9-985c-9d337baacd5e None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:04.400538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69004ac6-9122-4ad9-985c-9d337baacd5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:04.402040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69004ac6-9122-4ad9-985c-9d337baacd5e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:04.503989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 603/1205] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Aug 4 07:27:04 2022] POST /identity/v3/users => generated 251 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:27:06.548079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623acd5e-77bf-4e51-9f31-296d13af56f0 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:06.548677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623acd5e-77bf-4e51-9f31-296d13af56f0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:06.548677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623acd5e-77bf-4e51-9f31-296d13af56f0 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:06.550054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 603/1206] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:27:06.555268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb33a7-df29-42f5-86e0-d70a3db41005 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:06.555678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb33a7-df29-42f5-86e0-d70a3db41005 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:06.555825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb33a7-df29-42f5-86e0-d70a3db41005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:06.574535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-12eb33a7-df29-42f5-86e0-d70a3db41005 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:27:06.575934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-12eb33a7-df29-42f5-86e0-d70a3db41005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:27:06.641001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 604/1207] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:27:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 07:27:06.664411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623acd5e-77bf-4e51-9f31-296d13af56f0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:06.767941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623acd5e-77bf-4e51-9f31-296d13af56f0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:06.802020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e8450-de9e-4ce2-9acc-8ad08222f37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:06.802580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e8450-de9e-4ce2-9acc-8ad08222f37e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:06.802864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e8450-de9e-4ce2-9acc-8ad08222f37e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:06.803140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e8450-de9e-4ce2-9acc-8ad08222f37e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:06.813566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e8450-de9e-4ce2-9acc-8ad08222f37e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-156e8450-de9e-4ce2-9acc-8ad08222f37e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:27:06.818517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:06.819687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:06.820648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 604/1208] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:27:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:06.825929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb33a7-df29-42f5-86e0-d70a3db41005 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:06.864992 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:06.922499 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb33a7-df29-42f5-86e0-d70a3db41005 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:06.956791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1aa0c8-3e09-4015-a8d3-54a911c27c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:06.957416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1aa0c8-3e09-4015-a8d3-54a911c27c42 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:06.957699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1aa0c8-3e09-4015-a8d3-54a911c27c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:06.957977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1aa0c8-3e09-4015-a8d3-54a911c27c42 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:06.958692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1aa0c8-3e09-4015-a8d3-54a911c27c42 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:06.958999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1aa0c8-3e09-4015-a8d3-54a911c27c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:06.960417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1aa0c8-3e09-4015-a8d3-54a911c27c42 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:06.968982 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 605/1209] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:27:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:06.974807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e8450-de9e-4ce2-9acc-8ad08222f37e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:07.026865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:07.075878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e8450-de9e-4ce2-9acc-8ad08222f37e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:07.109376 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aee06-8ade-4a80-a13c-7873b06795a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:07.110050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aee06-8ade-4a80-a13c-7873b06795a0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:07.110332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aee06-8ade-4a80-a13c-7873b06795a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:07.110605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aee06-8ade-4a80-a13c-7873b06795a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:07.120568 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aee06-8ade-4a80-a13c-7873b06795a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:07.121985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aee06-8ade-4a80-a13c-7873b06795a0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-961aee06-8ade-4a80-a13c-7873b06795a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:07.135723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:07.136784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:07.137696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:07.137696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:07.137696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:07.137696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:07.137696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 605/1210] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:07.143243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1aa0c8-3e09-4015-a8d3-54a911c27c42 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:07.241624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:07.244664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1aa0c8-3e09-4015-a8d3-54a911c27c42 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:07.278132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922b32e-c8ca-4de3-bdd7-739fe8bdb58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:07.279441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b32e-c8ca-4de3-bdd7-739fe8bdb58f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:07.279941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b32e-c8ca-4de3-bdd7-739fe8bdb58f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:07.280259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b32e-c8ca-4de3-bdd7-739fe8bdb58f None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:07.280949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922b32e-c8ca-4de3-bdd7-739fe8bdb58f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:07.281259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922b32e-c8ca-4de3-bdd7-739fe8bdb58f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:07.282758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922b32e-c8ca-4de3-bdd7-739fe8bdb58f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:07.293884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 606/1211] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:07.299792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aee06-8ade-4a80-a13c-7873b06795a0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:07.399102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aee06-8ade-4a80-a13c-7873b06795a0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:07.400557 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:07.431883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de4eb5-c20f-40d6-8078-570f22543efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:07.432619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4eb5-c20f-40d6-8078-570f22543efa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:07.437438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4eb5-c20f-40d6-8078-570f22543efa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:07.437438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4eb5-c20f-40d6-8078-570f22543efa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:07.440450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de4eb5-c20f-40d6-8078-570f22543efa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:07.441918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de4eb5-c20f-40d6-8078-570f22543efa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-85de4eb5-c20f-40d6-8078-570f22543efa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:27:07.453696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:07.460659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:07.461870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:07.461870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:07.461870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:27:07.461870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:27:07.461870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:27:07.461870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:27:07.461870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:27:07.461870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:27:07.461870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:07.461870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:07.461870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 606/1212] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:07.462598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922b32e-c8ca-4de3-bdd7-739fe8bdb58f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:07.557934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922b32e-c8ca-4de3-bdd7-739fe8bdb58f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:07.591199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae7dbc-8c94-4a32-a2be-52bc7b2fb482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:07.591802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae7dbc-8c94-4a32-a2be-52bc7b2fb482 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:07.592109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae7dbc-8c94-4a32-a2be-52bc7b2fb482 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:07.592382 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae7dbc-8c94-4a32-a2be-52bc7b2fb482 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:07.593157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae7dbc-8c94-4a32-a2be-52bc7b2fb482 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:07.593457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae7dbc-8c94-4a32-a2be-52bc7b2fb482 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:07.594887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae7dbc-8c94-4a32-a2be-52bc7b2fb482 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:07.611531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 607/1213] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Aug 4 07:27:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:07.617396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de4eb5-c20f-40d6-8078-570f22543efa None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:07.716788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de4eb5-c20f-40d6-8078-570f22543efa None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:07.750275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75303113-9217-4985-9b4c-d0f996898dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:07.750887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75303113-9217-4985-9b4c-d0f996898dba None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:07.751217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75303113-9217-4985-9b4c-d0f996898dba None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:07.751497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75303113-9217-4985-9b4c-d0f996898dba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:07.780882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:07.816902 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75303113-9217-4985-9b4c-d0f996898dba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:07.818405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75303113-9217-4985-9b4c-d0f996898dba None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:07.915753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 607/1214] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:27:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:27:07.922510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae7dbc-8c94-4a32-a2be-52bc7b2fb482 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:07.941370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:08.021294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae7dbc-8c94-4a32-a2be-52bc7b2fb482 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:08.055895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d3250-64ab-4717-b8e3-80de4457fb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:08.056578 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d3250-64ab-4717-b8e3-80de4457fb04 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:08.056860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d3250-64ab-4717-b8e3-80de4457fb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:08.057135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d3250-64ab-4717-b8e3-80de4457fb04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:08.067195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d3250-64ab-4717-b8e3-80de4457fb04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:08.068676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d3250-64ab-4717-b8e3-80de4457fb04 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c32d3250-64ab-4717-b8e3-80de4457fb04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:08.083486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:08.084616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:08.085901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:08.085901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:08.085901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:08.085901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:08.085901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 608/1215] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:08.090623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75303113-9217-4985-9b4c-d0f996898dba None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:08.150895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:08.189135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75303113-9217-4985-9b4c-d0f996898dba None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:08.223098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa234be-6f7c-4c4a-9a3f-e7d3c7bb1589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:08.223704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa234be-6f7c-4c4a-9a3f-e7d3c7bb1589 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:08.224000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa234be-6f7c-4c4a-9a3f-e7d3c7bb1589 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:08.224279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa234be-6f7c-4c4a-9a3f-e7d3c7bb1589 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:08.224962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa234be-6f7c-4c4a-9a3f-e7d3c7bb1589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:08.225265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa234be-6f7c-4c4a-9a3f-e7d3c7bb1589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:08.226863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa234be-6f7c-4c4a-9a3f-e7d3c7bb1589 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:08.237630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 608/1216] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:08.243405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d3250-64ab-4717-b8e3-80de4457fb04 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:08.312614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:08.342181 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d3250-64ab-4717-b8e3-80de4457fb04 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:08.375462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd00368-84bb-4791-8ccc-c78f9fc6b932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:08.376184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd00368-84bb-4791-8ccc-c78f9fc6b932 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:08.376470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd00368-84bb-4791-8ccc-c78f9fc6b932 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:08.376748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd00368-84bb-4791-8ccc-c78f9fc6b932 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:08.383442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd00368-84bb-4791-8ccc-c78f9fc6b932 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:08.384989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd00368-84bb-4791-8ccc-c78f9fc6b932 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2cd00368-84bb-4791-8ccc-c78f9fc6b932 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:27:08.396769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:08.397933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:08.399010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:27:08.399010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:27:08.399010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:27:08.399010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:27:08.399010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:27:08.399010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:27:08.399010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:08.399010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:08.399010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 609/1217] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:27:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:08.403814 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa234be-6f7c-4c4a-9a3f-e7d3c7bb1589 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:08.500692 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa234be-6f7c-4c4a-9a3f-e7d3c7bb1589 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:08.522238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:08.533257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d05c07-c67c-4864-b0ef-eb741d6f9b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:08.533862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05c07-c67c-4864-b0ef-eb741d6f9b3f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:08.534141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05c07-c67c-4864-b0ef-eb741d6f9b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:08.534412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05c07-c67c-4864-b0ef-eb741d6f9b3f None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:08.535150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d05c07-c67c-4864-b0ef-eb741d6f9b3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:08.535453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d05c07-c67c-4864-b0ef-eb741d6f9b3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:08.536939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d05c07-c67c-4864-b0ef-eb741d6f9b3f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:08.548481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 609/1218] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:27:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:08.554585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd00368-84bb-4791-8ccc-c78f9fc6b932 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:08.655689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd00368-84bb-4791-8ccc-c78f9fc6b932 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:08.683423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:08.690160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6e1a2-066c-4518-ac36-c716f922e22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:08.690761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6e1a2-066c-4518-ac36-c716f922e22f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:08.691039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6e1a2-066c-4518-ac36-c716f922e22f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:08.691310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6e1a2-066c-4518-ac36-c716f922e22f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:08.692106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6e1a2-066c-4518-ac36-c716f922e22f None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=affaeff3cd294584b7238909e0457f80) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:08.692412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6e1a2-066c-4518-ac36-c716f922e22f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:08.693921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6e1a2-066c-4518-ac36-c716f922e22f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:08.699284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 610/1219] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:27:08 2022] GET /identity/v3/role_assignments?user.id=affaeff3cd294584b7238909e0457f80&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:10.729491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39823c85-03b7-451a-8a01-d6c8e3bc3752 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:10.730087 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39823c85-03b7-451a-8a01-d6c8e3bc3752 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:10.730087 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39823c85-03b7-451a-8a01-d6c8e3bc3752 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:10.731700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 610/1220] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:27:10.737384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a908-8ce7-4782-8522-5bb4252ee41a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:10.737793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a908-8ce7-4782-8522-5bb4252ee41a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:10.737952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a908-8ce7-4782-8522-5bb4252ee41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:10.755789 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1d36a908-8ce7-4782-8522-5bb4252ee41a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:27:10.757016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1d36a908-8ce7-4782-8522-5bb4252ee41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:27:10.822662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 611/1221] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:27:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:27:10.846293 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39823c85-03b7-451a-8a01-d6c8e3bc3752 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:10.946912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39823c85-03b7-451a-8a01-d6c8e3bc3752 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:10.981717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80331-8383-4818-9204-76d26232c491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:10.982292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80331-8383-4818-9204-76d26232c491 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:10.982574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80331-8383-4818-9204-76d26232c491 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:10.982850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80331-8383-4818-9204-76d26232c491 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:10.992621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f80331-8383-4818-9204-76d26232c491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-39f80331-8383-4818-9204-76d26232c491 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:27:10.997674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:11.002941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:11.004009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 611/1222] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:27:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:11.005330 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36a908-8ce7-4782-8522-5bb4252ee41a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:11.103278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36a908-8ce7-4782-8522-5bb4252ee41a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:11.125629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:11.135825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab00d9a-476a-42cb-9781-7c76230694c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:11.136470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab00d9a-476a-42cb-9781-7c76230694c4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:11.136751 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab00d9a-476a-42cb-9781-7c76230694c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:11.137026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab00d9a-476a-42cb-9781-7c76230694c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:11.137728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab00d9a-476a-42cb-9781-7c76230694c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:11.138745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab00d9a-476a-42cb-9781-7c76230694c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:11.140148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab00d9a-476a-42cb-9781-7c76230694c4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:11.147709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 612/1223] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:27:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:11.154180 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80331-8383-4818-9204-76d26232c491 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:11.196570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:11.251466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80331-8383-4818-9204-76d26232c491 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:11.285446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7712a1-3e70-4ea4-9a32-3656b2c4cfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:11.286077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7712a1-3e70-4ea4-9a32-3656b2c4cfc3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:11.286362 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7712a1-3e70-4ea4-9a32-3656b2c4cfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:11.286639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7712a1-3e70-4ea4-9a32-3656b2c4cfc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:11.296628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7712a1-3e70-4ea4-9a32-3656b2c4cfc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:11.298053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7712a1-3e70-4ea4-9a32-3656b2c4cfc3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6a7712a1-3e70-4ea4-9a32-3656b2c4cfc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:11.313905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:11.314968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:11.316006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:11.316006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:11.316006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:11.316006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:11.316006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 612/1224] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:11.320835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab00d9a-476a-42cb-9781-7c76230694c4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:11.420502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab00d9a-476a-42cb-9781-7c76230694c4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:11.453553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95983f98-a111-4d7a-b7e0-37fb20f1d30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:11.454162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95983f98-a111-4d7a-b7e0-37fb20f1d30a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:11.454446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95983f98-a111-4d7a-b7e0-37fb20f1d30a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:11.456622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95983f98-a111-4d7a-b7e0-37fb20f1d30a None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:11.456622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95983f98-a111-4d7a-b7e0-37fb20f1d30a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:11.456622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95983f98-a111-4d7a-b7e0-37fb20f1d30a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:11.457075 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95983f98-a111-4d7a-b7e0-37fb20f1d30a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:11.468528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 613/1225] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:11.474284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7712a1-3e70-4ea4-9a32-3656b2c4cfc3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:11.566342 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:11.572867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7712a1-3e70-4ea4-9a32-3656b2c4cfc3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:11.605670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fdc24-6101-4ab8-a1c8-74a3c4fbf449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:11.606304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fdc24-6101-4ab8-a1c8-74a3c4fbf449 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:11.607194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fdc24-6101-4ab8-a1c8-74a3c4fbf449 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:11.607519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fdc24-6101-4ab8-a1c8-74a3c4fbf449 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:11.614712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fdc24-6101-4ab8-a1c8-74a3c4fbf449 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:11.616188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fdc24-6101-4ab8-a1c8-74a3c4fbf449 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-382fdc24-6101-4ab8-a1c8-74a3c4fbf449 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:27:11.628161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:11.629173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:11.630179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:11.630179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:11.630179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:27:11.630179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:27:11.630179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:27:11.630179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:27:11.630179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:27:11.630179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:27:11.630179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:11.630179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:11.630179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 613/1226] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:11.635435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95983f98-a111-4d7a-b7e0-37fb20f1d30a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:11.680249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:11.730478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95983f98-a111-4d7a-b7e0-37fb20f1d30a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:11.764353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a989c-98fb-451c-9c59-001f4f92109c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:11.765199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a989c-98fb-451c-9c59-001f4f92109c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:11.765298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a989c-98fb-451c-9c59-001f4f92109c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:11.765552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a989c-98fb-451c-9c59-001f4f92109c None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:11.766583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a989c-98fb-451c-9c59-001f4f92109c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:11.767135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a989c-98fb-451c-9c59-001f4f92109c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:11.768916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a989c-98fb-451c-9c59-001f4f92109c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:11.786395 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 614/1227] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Aug 4 07:27:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:11.792122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fdc24-6101-4ab8-a1c8-74a3c4fbf449 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:11.892512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fdc24-6101-4ab8-a1c8-74a3c4fbf449 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:11.925765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c00aa0-71b6-4c97-8f9a-896f06739c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:11.926396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c00aa0-71b6-4c97-8f9a-896f06739c34 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:11.926682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c00aa0-71b6-4c97-8f9a-896f06739c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:11.926964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c00aa0-71b6-4c97-8f9a-896f06739c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:11.937822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c00aa0-71b6-4c97-8f9a-896f06739c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:11.939254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c00aa0-71b6-4c97-8f9a-896f06739c34 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-75c00aa0-71b6-4c97-8f9a-896f06739c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:11.953423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:11.954492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:11.955395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:11.955395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:11.955395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:11.955395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:11.955395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 614/1228] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:27:11.960390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a989c-98fb-451c-9c59-001f4f92109c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:12.059704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a989c-98fb-451c-9c59-001f4f92109c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:12.062012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:12.093229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db4fee-9551-401d-b967-40a7ea91d312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:12.093838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db4fee-9551-401d-b967-40a7ea91d312 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:12.094117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db4fee-9551-401d-b967-40a7ea91d312 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:12.094399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db4fee-9551-401d-b967-40a7ea91d312 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:12.095065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db4fee-9551-401d-b967-40a7ea91d312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:12.095366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db4fee-9551-401d-b967-40a7ea91d312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:12.096749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db4fee-9551-401d-b967-40a7ea91d312 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:12.108875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 615/1229] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:12.114641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c00aa0-71b6-4c97-8f9a-896f06739c34 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:12.124212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:12.215368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c00aa0-71b6-4c97-8f9a-896f06739c34 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:12.249203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e69ba-aed1-4bb1-b0fb-29edcbdd3372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:12.249836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e69ba-aed1-4bb1-b0fb-29edcbdd3372 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:12.250119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e69ba-aed1-4bb1-b0fb-29edcbdd3372 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:12.250392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e69ba-aed1-4bb1-b0fb-29edcbdd3372 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:12.257163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e69ba-aed1-4bb1-b0fb-29edcbdd3372 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:12.262720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e69ba-aed1-4bb1-b0fb-29edcbdd3372 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c62e69ba-aed1-4bb1-b0fb-29edcbdd3372 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:27:12.269996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:12.271103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:12.272473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:27:12.272473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:27:12.272473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:27:12.272473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:27:12.272473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:27:12.272473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:27:12.272473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:12.272473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:12.272473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 615/1230] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:27:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:12.277019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db4fee-9551-401d-b967-40a7ea91d312 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:12.372951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db4fee-9551-401d-b967-40a7ea91d312 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:12.405566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79bd32-9e07-44a8-b013-33dc3676bf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:12.406162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79bd32-9e07-44a8-b013-33dc3676bf8e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:12.406440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79bd32-9e07-44a8-b013-33dc3676bf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:12.406719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79bd32-9e07-44a8-b013-33dc3676bf8e None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:12.407450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79bd32-9e07-44a8-b013-33dc3676bf8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:12.407753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79bd32-9e07-44a8-b013-33dc3676bf8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:12.409216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79bd32-9e07-44a8-b013-33dc3676bf8e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:12.420532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 616/1231] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:27:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:12.426438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e69ba-aed1-4bb1-b0fb-29edcbdd3372 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:12.491771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:12.526605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e69ba-aed1-4bb1-b0fb-29edcbdd3372 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:12.559616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c683d8-52b6-4f80-b5ff-8365fdbb5b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:12.560514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c683d8-52b6-4f80-b5ff-8365fdbb5b45 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:12.560790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c683d8-52b6-4f80-b5ff-8365fdbb5b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:12.561063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c683d8-52b6-4f80-b5ff-8365fdbb5b45 None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/affaeff3cd294584b7238909e0457f80/roles/88637a5a7c9d47aaacf803a09abc1755` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:12.576230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c683d8-52b6-4f80-b5ff-8365fdbb5b45 None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=affaeff3cd294584b7238909e0457f80, role_id=88637a5a7c9d47aaacf803a09abc1755)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:12.577700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c683d8-52b6-4f80-b5ff-8365fdbb5b45 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:12.660190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 616/1232] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:27:12 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/affaeff3cd294584b7238909e0457f80/roles/88637a5a7c9d47aaacf803a09abc1755 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:27:14.627591 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d847f-dc0f-4313-9f3f-2f644b46c3fe None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:14.628152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d847f-dc0f-4313-9f3f-2f644b46c3fe None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:14.628246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d847f-dc0f-4313-9f3f-2f644b46c3fe None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:14.629592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 617/1233] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:27:14.634849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7b49f-5704-457f-be6a-e8b9e00aed52 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:14.635263 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7b49f-5704-457f-be6a-e8b9e00aed52 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:14.635417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7b49f-5704-457f-be6a-e8b9e00aed52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:14.654400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4dc7b49f-5704-457f-be6a-e8b9e00aed52 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:27:14.655594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4dc7b49f-5704-457f-be6a-e8b9e00aed52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:27:14.720691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 617/1234] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:27:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:27:14.743965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d847f-dc0f-4313-9f3f-2f644b46c3fe None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:14.769740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:14.844886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d847f-dc0f-4313-9f3f-2f644b46c3fe None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:14.879657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821d834-a50d-42e3-a199-99579087609c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:14.880253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821d834-a50d-42e3-a199-99579087609c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:14.880538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821d834-a50d-42e3-a199-99579087609c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:14.880813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821d834-a50d-42e3-a199-99579087609c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:14.891279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821d834-a50d-42e3-a199-99579087609c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d821d834-a50d-42e3-a199-99579087609c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:27:14.896119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:14.901862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:14.902946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 618/1235] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Aug 4 07:27:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:14.904582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7b49f-5704-457f-be6a-e8b9e00aed52 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:14.950896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:15.002074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7b49f-5704-457f-be6a-e8b9e00aed52 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:15.034927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cdfe9-9358-4aa7-ba66-717ba4fb8206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:15.035541 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cdfe9-9358-4aa7-ba66-717ba4fb8206 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:15.035820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cdfe9-9358-4aa7-ba66-717ba4fb8206 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:15.036127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cdfe9-9358-4aa7-ba66-717ba4fb8206 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:15.036833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cdfe9-9358-4aa7-ba66-717ba4fb8206 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:15.037135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cdfe9-9358-4aa7-ba66-717ba4fb8206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:15.038481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cdfe9-9358-4aa7-ba66-717ba4fb8206 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:15.046930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 618/1236] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Aug 4 07:27:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:15.052902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821d834-a50d-42e3-a199-99579087609c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:15.136448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:15.152379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821d834-a50d-42e3-a199-99579087609c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:15.185593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62cc49-c3bd-4179-b748-ddb9445fbe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:15.186238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62cc49-c3bd-4179-b748-ddb9445fbe6a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:15.186518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62cc49-c3bd-4179-b748-ddb9445fbe6a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:15.186792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62cc49-c3bd-4179-b748-ddb9445fbe6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:15.196956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62cc49-c3bd-4179-b748-ddb9445fbe6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:15.198939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62cc49-c3bd-4179-b748-ddb9445fbe6a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4a62cc49-c3bd-4179-b748-ddb9445fbe6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:15.214265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:15.215304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:15.216360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:15.216360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:15.216360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:15.216360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:15.216360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 619/1237] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:15.221699 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cdfe9-9358-4aa7-ba66-717ba4fb8206 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:15.319553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cdfe9-9358-4aa7-ba66-717ba4fb8206 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:15.328820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:15.353137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3682-905e-409a-9229-c55643408672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:15.353754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3682-905e-409a-9229-c55643408672 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:15.354033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3682-905e-409a-9229-c55643408672 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:15.354306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3682-905e-409a-9229-c55643408672 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:15.355050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a3682-905e-409a-9229-c55643408672 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:15.355353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a3682-905e-409a-9229-c55643408672 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:15.357301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a3682-905e-409a-9229-c55643408672 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:15.368956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 619/1238] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:15.375242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62cc49-c3bd-4179-b748-ddb9445fbe6a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:15.474175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62cc49-c3bd-4179-b748-ddb9445fbe6a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:15.509284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1eac37-1d88-49c0-acca-2b2b3a8566a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:15.509921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eac37-1d88-49c0-acca-2b2b3a8566a2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:15.510207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eac37-1d88-49c0-acca-2b2b3a8566a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:15.510485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eac37-1d88-49c0-acca-2b2b3a8566a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:15.513869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:15.518698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1eac37-1d88-49c0-acca-2b2b3a8566a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:15.520209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1eac37-1d88-49c0-acca-2b2b3a8566a2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8c1eac37-1d88-49c0-acca-2b2b3a8566a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:27:15.531734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:15.532869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:15.533879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:15.533879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:15.533879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:27:15.533879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:27:15.533879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:27:15.533879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:27:15.533879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:27:15.533879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:27:15.533879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:15.533879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:15.533879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 620/1239] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:27:15.538734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3682-905e-409a-9229-c55643408672 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:15.636030 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3682-905e-409a-9229-c55643408672 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:15.670115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d989251-1d14-4dc9-b1ee-e7a60b874f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:15.670730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d989251-1d14-4dc9-b1ee-e7a60b874f50 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:15.671015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d989251-1d14-4dc9-b1ee-e7a60b874f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:15.671294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d989251-1d14-4dc9-b1ee-e7a60b874f50 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:15.672115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d989251-1d14-4dc9-b1ee-e7a60b874f50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:15.672427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d989251-1d14-4dc9-b1ee-e7a60b874f50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:15.673879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d989251-1d14-4dc9-b1ee-e7a60b874f50 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:15.690455 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 620/1240] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Aug 4 07:27:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:15.696309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1eac37-1d88-49c0-acca-2b2b3a8566a2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:15.796220 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1eac37-1d88-49c0-acca-2b2b3a8566a2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:15.828709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c67e19-248e-4436-b06f-59e688c8ba3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:15.829309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c67e19-248e-4436-b06f-59e688c8ba3a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:15.829588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c67e19-248e-4436-b06f-59e688c8ba3a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:15.829859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c67e19-248e-4436-b06f-59e688c8ba3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:15.890369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:15.897029 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c67e19-248e-4436-b06f-59e688c8ba3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:15.898482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c67e19-248e-4436-b06f-59e688c8ba3a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:15.992926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 621/1241] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:27:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:27:15.999522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d989251-1d14-4dc9-b1ee-e7a60b874f50 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:16.027258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:16.097684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d989251-1d14-4dc9-b1ee-e7a60b874f50 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:16.132242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00609aa7-dfd8-4560-9e2e-3b01689e30d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:16.134649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00609aa7-dfd8-4560-9e2e-3b01689e30d2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:16.134649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00609aa7-dfd8-4560-9e2e-3b01689e30d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:16.134649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00609aa7-dfd8-4560-9e2e-3b01689e30d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:16.144152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00609aa7-dfd8-4560-9e2e-3b01689e30d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:16.145581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00609aa7-dfd8-4560-9e2e-3b01689e30d2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-00609aa7-dfd8-4560-9e2e-3b01689e30d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:16.160614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:16.161701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:16.162860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:16.162860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:16.162860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:16.162860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:16.162860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 621/1242] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:16.167648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c67e19-248e-4436-b06f-59e688c8ba3a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:16.259487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:16.266067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c67e19-248e-4436-b06f-59e688c8ba3a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:16.299303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39c05e-d01f-4d26-8f21-f4ebec5ba4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:16.299905 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39c05e-d01f-4d26-8f21-f4ebec5ba4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:16.300219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39c05e-d01f-4d26-8f21-f4ebec5ba4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:16.300498 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39c05e-d01f-4d26-8f21-f4ebec5ba4a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:16.301184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39c05e-d01f-4d26-8f21-f4ebec5ba4a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:16.301490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39c05e-d01f-4d26-8f21-f4ebec5ba4a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:16.302897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39c05e-d01f-4d26-8f21-f4ebec5ba4a4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:16.314224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 622/1243] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:16.320082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00609aa7-dfd8-4560-9e2e-3b01689e30d2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:16.396979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:16.417554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00609aa7-dfd8-4560-9e2e-3b01689e30d2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:16.448994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595912f5-0e76-41cb-927f-06e0066c98fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:16.449621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595912f5-0e76-41cb-927f-06e0066c98fc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:16.449911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595912f5-0e76-41cb-927f-06e0066c98fc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:16.450848 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595912f5-0e76-41cb-927f-06e0066c98fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:16.457927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595912f5-0e76-41cb-927f-06e0066c98fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:16.459357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595912f5-0e76-41cb-927f-06e0066c98fc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-595912f5-0e76-41cb-927f-06e0066c98fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:27:16.471118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:16.472229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:16.473389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:27:16.473389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:27:16.473389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:27:16.473389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:27:16.473389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:27:16.473389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:27:16.473389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:16.473389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:16.473389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 622/1244] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:27:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:16.478649 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39c05e-d01f-4d26-8f21-f4ebec5ba4a4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:16.578094 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39c05e-d01f-4d26-8f21-f4ebec5ba4a4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:16.610705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9100f7d-2955-4013-8e21-3e11afd1d532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:16.611433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9100f7d-2955-4013-8e21-3e11afd1d532 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:16.611722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9100f7d-2955-4013-8e21-3e11afd1d532 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:16.612020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9100f7d-2955-4013-8e21-3e11afd1d532 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:16.612771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9100f7d-2955-4013-8e21-3e11afd1d532 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:16.613082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9100f7d-2955-4013-8e21-3e11afd1d532 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:16.614513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9100f7d-2955-4013-8e21-3e11afd1d532 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:16.626607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 623/1245] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:27:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:16.632826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595912f5-0e76-41cb-927f-06e0066c98fc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:16.731948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595912f5-0e76-41cb-927f-06e0066c98fc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:16.765019 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeec035-802c-4641-9b26-6521897bfa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:16.765621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeec035-802c-4641-9b26-6521897bfa05 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:16.765907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeec035-802c-4641-9b26-6521897bfa05 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:16.766282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeec035-802c-4641-9b26-6521897bfa05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:16.767207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeec035-802c-4641-9b26-6521897bfa05 None admin] RBAC: Adding query filter params (role.id=88637a5a7c9d47aaacf803a09abc1755, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=affaeff3cd294584b7238909e0457f80) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:16.767520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeec035-802c-4641-9b26-6521897bfa05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:16.768974 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeec035-802c-4641-9b26-6521897bfa05 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:16.774782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 623/1246] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:27:16 2022] GET /identity/v3/role_assignments?user.id=affaeff3cd294584b7238909e0457f80&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=88637a5a7c9d47aaacf803a09abc1755 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:18.830537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7a8e8-7d04-4aeb-bd81-32932c6d1ad2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:18.831326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7a8e8-7d04-4aeb-bd81-32932c6d1ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:18.831326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7a8e8-7d04-4aeb-bd81-32932c6d1ad2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:18.832582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 624/1247] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:27:18.837864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ed750-449a-41f2-a019-58f780f94166 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:18.838304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ed750-449a-41f2-a019-58f780f94166 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:18.838459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ed750-449a-41f2-a019-58f780f94166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:18.844950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:18.857266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-587ed750-449a-41f2-a019-58f780f94166 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:27:18.858454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-587ed750-449a-41f2-a019-58f780f94166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:27:18.922531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 624/1248] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:27:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:27:18.946074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7a8e8-7d04-4aeb-bd81-32932c6d1ad2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:18.966409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:19.050039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7a8e8-7d04-4aeb-bd81-32932c6d1ad2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:19.085134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f31ea5-30f9-4c08-8bd0-5e27a906c160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:19.085707 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f31ea5-30f9-4c08-8bd0-5e27a906c160 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:19.085994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f31ea5-30f9-4c08-8bd0-5e27a906c160 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:19.086273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f31ea5-30f9-4c08-8bd0-5e27a906c160 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:19.096388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f31ea5-30f9-4c08-8bd0-5e27a906c160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a5f31ea5-30f9-4c08-8bd0-5e27a906c160 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:27:19.101622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:19.102779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:19.103887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 625/1249] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:27:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:19.111881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ed750-449a-41f2-a019-58f780f94166 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:19.208705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ed750-449a-41f2-a019-58f780f94166 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:19.237962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:19.242943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddded687-1bf0-434a-901a-ac95374d74ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:19.243665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddded687-1bf0-434a-901a-ac95374d74ef None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:19.243954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddded687-1bf0-434a-901a-ac95374d74ef None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:19.244266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddded687-1bf0-434a-901a-ac95374d74ef None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:19.244982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddded687-1bf0-434a-901a-ac95374d74ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:19.245288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddded687-1bf0-434a-901a-ac95374d74ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:19.246648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddded687-1bf0-434a-901a-ac95374d74ef None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:19.255661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 625/1250] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:27:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:19.270056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f31ea5-30f9-4c08-8bd0-5e27a906c160 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:19.336535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:19.358573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f31ea5-30f9-4c08-8bd0-5e27a906c160 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:19.390531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4342f-7b56-442d-9e73-b0b4a0a605fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:19.391140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4342f-7b56-442d-9e73-b0b4a0a605fc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:19.391418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4342f-7b56-442d-9e73-b0b4a0a605fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:19.391689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4342f-7b56-442d-9e73-b0b4a0a605fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:19.401062 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4342f-7b56-442d-9e73-b0b4a0a605fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:19.402478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4342f-7b56-442d-9e73-b0b4a0a605fc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-76e4342f-7b56-442d-9e73-b0b4a0a605fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:19.416683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:19.417812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:19.419416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:19.419416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:19.419416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:19.419416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:19.419416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 626/1251] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:19.424413 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddded687-1bf0-434a-901a-ac95374d74ef None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:19.522136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddded687-1bf0-434a-901a-ac95374d74ef None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:19.554719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ebd19-ec75-4aa6-b38f-aa20e4be334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:19.555316 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ebd19-ec75-4aa6-b38f-aa20e4be334a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:19.555594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ebd19-ec75-4aa6-b38f-aa20e4be334a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:19.555874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ebd19-ec75-4aa6-b38f-aa20e4be334a None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:19.556580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ebd19-ec75-4aa6-b38f-aa20e4be334a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:19.556883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ebd19-ec75-4aa6-b38f-aa20e4be334a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:19.558239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ebd19-ec75-4aa6-b38f-aa20e4be334a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:19.569428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 626/1252] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:19.575250 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4342f-7b56-442d-9e73-b0b4a0a605fc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:19.674524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4342f-7b56-442d-9e73-b0b4a0a605fc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:19.699480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:19.707626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef8cbb-3f22-42f8-ae53-84c5f6f4d0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:19.708296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef8cbb-3f22-42f8-ae53-84c5f6f4d0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:19.708581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef8cbb-3f22-42f8-ae53-84c5f6f4d0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:19.708860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef8cbb-3f22-42f8-ae53-84c5f6f4d0a0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:19.716669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef8cbb-3f22-42f8-ae53-84c5f6f4d0a0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:19.718124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef8cbb-3f22-42f8-ae53-84c5f6f4d0a0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a7ef8cbb-3f22-42f8-ae53-84c5f6f4d0a0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:27:19.729318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:19.730518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:19.732692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:19.732692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:19.732692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:27:19.732692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:27:19.732692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:27:19.732692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:27:19.732692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:27:19.732692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:27:19.732692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:19.732692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:19.732692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 627/1253] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:19.738605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ebd19-ec75-4aa6-b38f-aa20e4be334a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:19.789888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:19.833200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ebd19-ec75-4aa6-b38f-aa20e4be334a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:19.867972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f473b-c0ce-406a-98e4-ec038498e8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:19.868618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f473b-c0ce-406a-98e4-ec038498e8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:19.868899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f473b-c0ce-406a-98e4-ec038498e8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:19.869172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f473b-c0ce-406a-98e4-ec038498e8d9 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:19.870005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f473b-c0ce-406a-98e4-ec038498e8d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:19.870310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f473b-c0ce-406a-98e4-ec038498e8d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:19.871844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f473b-c0ce-406a-98e4-ec038498e8d9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:19.889156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 627/1254] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Aug 4 07:27:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:19.895117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef8cbb-3f22-42f8-ae53-84c5f6f4d0a0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:19.998551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef8cbb-3f22-42f8-ae53-84c5f6f4d0a0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:20.031580 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aec9ef-b825-4d79-b09f-08fc61757bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:20.032226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec9ef-b825-4d79-b09f-08fc61757bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:20.032555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec9ef-b825-4d79-b09f-08fc61757bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:20.032838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec9ef-b825-4d79-b09f-08fc61757bb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:20.088196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:20.099821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aec9ef-b825-4d79-b09f-08fc61757bb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:20.101543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aec9ef-b825-4d79-b09f-08fc61757bb2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:20.198231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 628/1255] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:27:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:27:20.204227 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f473b-c0ce-406a-98e4-ec038498e8d9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:20.301307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f473b-c0ce-406a-98e4-ec038498e8d9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:20.314659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:20.335286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85429d2c-f868-41d0-a487-cc2653db1809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:20.335920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85429d2c-f868-41d0-a487-cc2653db1809 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:20.336232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85429d2c-f868-41d0-a487-cc2653db1809 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:20.336508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85429d2c-f868-41d0-a487-cc2653db1809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:20.346415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85429d2c-f868-41d0-a487-cc2653db1809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:20.347848 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85429d2c-f868-41d0-a487-cc2653db1809 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-85429d2c-f868-41d0-a487-cc2653db1809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:20.362853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:20.363942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:20.365117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:20.365117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:20.365117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:20.365117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:20.365117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 628/1256] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:20.371447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aec9ef-b825-4d79-b09f-08fc61757bb2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:20.459685 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:20.470649 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aec9ef-b825-4d79-b09f-08fc61757bb2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:20.503860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c30bc-791c-41d3-a70b-3e94fcb5dec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:20.504494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c30bc-791c-41d3-a70b-3e94fcb5dec4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:20.504781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c30bc-791c-41d3-a70b-3e94fcb5dec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:20.505059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c30bc-791c-41d3-a70b-3e94fcb5dec4 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:20.505741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c30bc-791c-41d3-a70b-3e94fcb5dec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:20.506047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c30bc-791c-41d3-a70b-3e94fcb5dec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:20.507439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c30bc-791c-41d3-a70b-3e94fcb5dec4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:20.518869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 629/1257] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:20.524833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85429d2c-f868-41d0-a487-cc2653db1809 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:20.627144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85429d2c-f868-41d0-a487-cc2653db1809 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:20.659809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaae533-857b-4b13-b20f-5fcc302de912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:20.660477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaae533-857b-4b13-b20f-5fcc302de912 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:20.660760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaae533-857b-4b13-b20f-5fcc302de912 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:20.661036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaae533-857b-4b13-b20f-5fcc302de912 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:20.667641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaae533-857b-4b13-b20f-5fcc302de912 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:20.669769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaae533-857b-4b13-b20f-5fcc302de912 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8aaae533-857b-4b13-b20f-5fcc302de912 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:27:20.680972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:20.681882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:20.683026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:27:20.683026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:27:20.683026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:27:20.683026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:27:20.683026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:27:20.683026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:27:20.683026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:20.683026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:20.683026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 629/1258] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:27:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:20.688699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c30bc-791c-41d3-a70b-3e94fcb5dec4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:20.785655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c30bc-791c-41d3-a70b-3e94fcb5dec4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:20.818775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefff3d1-263a-439b-8e38-84d370c525c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:20.819378 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefff3d1-263a-439b-8e38-84d370c525c7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:20.819662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefff3d1-263a-439b-8e38-84d370c525c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:20.819936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefff3d1-263a-439b-8e38-84d370c525c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:20.820703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefff3d1-263a-439b-8e38-84d370c525c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:20.821011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefff3d1-263a-439b-8e38-84d370c525c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:20.822433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefff3d1-263a-439b-8e38-84d370c525c7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:20.834562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 630/1259] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:27:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:20.840718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaae533-857b-4b13-b20f-5fcc302de912 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:20.866619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:20.939390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaae533-857b-4b13-b20f-5fcc302de912 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:20.973596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbfa54-91f9-4b8c-9069-5ca2cd314d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:20.982779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbfa54-91f9-4b8c-9069-5ca2cd314d68 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:20.982779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbfa54-91f9-4b8c-9069-5ca2cd314d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:20.982779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbfa54-91f9-4b8c-9069-5ca2cd314d68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:20.982779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbfa54-91f9-4b8c-9069-5ca2cd314d68 None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=affaeff3cd294584b7238909e0457f80) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:20.982779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbfa54-91f9-4b8c-9069-5ca2cd314d68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:20.982779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbfa54-91f9-4b8c-9069-5ca2cd314d68 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:20.983501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 630/1260] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:27:20 2022] GET /identity/v3/role_assignments?user.id=affaeff3cd294584b7238909e0457f80&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:22.970851 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4654c8-51ea-4b9d-abd6-dbe196db9573 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:22.971301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4654c8-51ea-4b9d-abd6-dbe196db9573 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:22.971404 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4654c8-51ea-4b9d-abd6-dbe196db9573 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:22.972786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 631/1261] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:27:22.978405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbbea3-6987-4858-ba1b-e9a907efe406 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:22.986130 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbbea3-6987-4858-ba1b-e9a907efe406 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:22.986130 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbbea3-6987-4858-ba1b-e9a907efe406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:22.997829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ddbbbea3-6987-4858-ba1b-e9a907efe406 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:27:22.998997 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ddbbbea3-6987-4858-ba1b-e9a907efe406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:27:23.066308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 631/1262] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:27:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:27:23.088944 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4654c8-51ea-4b9d-abd6-dbe196db9573 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:23.100229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:23.187531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4654c8-51ea-4b9d-abd6-dbe196db9573 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:23.221397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9f540-49d6-4e59-bde5-85b66241b24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:23.221949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9f540-49d6-4e59-bde5-85b66241b24a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:23.222227 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9f540-49d6-4e59-bde5-85b66241b24a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:23.222500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9f540-49d6-4e59-bde5-85b66241b24a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:23.232834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9f540-49d6-4e59-bde5-85b66241b24a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5ee9f540-49d6-4e59-bde5-85b66241b24a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:27:23.237691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:23.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:23.239754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 632/1263] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:27:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:23.244998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbbea3-6987-4858-ba1b-e9a907efe406 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:23.264354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:23.342051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbbea3-6987-4858-ba1b-e9a907efe406 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:23.374358 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9be0e-75cd-484d-918d-a2812e509199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:23.374966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9be0e-75cd-484d-918d-a2812e509199 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:23.375247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9be0e-75cd-484d-918d-a2812e509199 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:23.375521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9be0e-75cd-484d-918d-a2812e509199 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:23.376260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9be0e-75cd-484d-918d-a2812e509199 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:23.376565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9be0e-75cd-484d-918d-a2812e509199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:23.377919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9be0e-75cd-484d-918d-a2812e509199 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:23.385536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 632/1264] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:27:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:23.392263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9f540-49d6-4e59-bde5-85b66241b24a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:23.456986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:23.488744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9f540-49d6-4e59-bde5-85b66241b24a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:23.522978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd191b-fbd5-4272-906c-67d29e51164e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:23.523617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd191b-fbd5-4272-906c-67d29e51164e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:23.523900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd191b-fbd5-4272-906c-67d29e51164e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:23.524207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd191b-fbd5-4272-906c-67d29e51164e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:23.534846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd191b-fbd5-4272-906c-67d29e51164e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:23.536372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd191b-fbd5-4272-906c-67d29e51164e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-92dd191b-fbd5-4272-906c-67d29e51164e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:23.550994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:23.552167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:23.553288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:23.553288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:23.553288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:23.553288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:23.553288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 633/1265] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:23.558124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9be0e-75cd-484d-918d-a2812e509199 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:23.640900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:23.659395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9be0e-75cd-484d-918d-a2812e509199 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:23.692852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee813b7-1503-40ed-b645-da215a915739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:23.693394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee813b7-1503-40ed-b645-da215a915739 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:23.693677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee813b7-1503-40ed-b645-da215a915739 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:23.693951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee813b7-1503-40ed-b645-da215a915739 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:23.694637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee813b7-1503-40ed-b645-da215a915739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:23.694947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee813b7-1503-40ed-b645-da215a915739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:23.696320 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee813b7-1503-40ed-b645-da215a915739 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:23.708144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 633/1266] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:23.713968 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd191b-fbd5-4272-906c-67d29e51164e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:23.813177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd191b-fbd5-4272-906c-67d29e51164e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:23.832672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:23.846526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1dfcb-ef05-4fab-b27d-e5d1595b518a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:23.847157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1dfcb-ef05-4fab-b27d-e5d1595b518a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:23.847440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1dfcb-ef05-4fab-b27d-e5d1595b518a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:23.847715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1dfcb-ef05-4fab-b27d-e5d1595b518a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:23.855076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1dfcb-ef05-4fab-b27d-e5d1595b518a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:23.856663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1dfcb-ef05-4fab-b27d-e5d1595b518a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7fc1dfcb-ef05-4fab-b27d-e5d1595b518a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:27:23.867432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:23.868565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:23.869761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:23.869761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:23.869761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:27:23.869761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:27:23.869761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:27:23.869761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:27:23.869761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:27:23.869761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:27:23.869761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:23.869761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:23.869761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 634/1267] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 07:27:23.876935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee813b7-1503-40ed-b645-da215a915739 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:23.977626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee813b7-1503-40ed-b645-da215a915739 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:24.011851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca7739-7438-47c0-b934-9b2c9eb248ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:24.013221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca7739-7438-47c0-b934-9b2c9eb248ec None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:24.013707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca7739-7438-47c0-b934-9b2c9eb248ec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:24.013993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca7739-7438-47c0-b934-9b2c9eb248ec None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:24.014793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca7739-7438-47c0-b934-9b2c9eb248ec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:24.015106 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca7739-7438-47c0-b934-9b2c9eb248ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:24.016611 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca7739-7438-47c0-b934-9b2c9eb248ec None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:24.025120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:24.033474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 634/1268] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Aug 4 07:27:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:24.039361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1dfcb-ef05-4fab-b27d-e5d1595b518a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:24.141480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1dfcb-ef05-4fab-b27d-e5d1595b518a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:24.173975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a3701-16b8-407c-9aa8-092c9ce58f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:24.174728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3701-16b8-407c-9aa8-092c9ce58f18 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:24.174881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3701-16b8-407c-9aa8-092c9ce58f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:24.175157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3701-16b8-407c-9aa8-092c9ce58f18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:24.187331 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a3701-16b8-407c-9aa8-092c9ce58f18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:24.188814 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a3701-16b8-407c-9aa8-092c9ce58f18 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a48a3701-16b8-407c-9aa8-092c9ce58f18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:24.203624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:24.204835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:24.207007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:24.207007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:24.207007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:24.207007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:24.207007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 635/1269] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:24.211714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca7739-7438-47c0-b934-9b2c9eb248ec None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:24.310913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca7739-7438-47c0-b934-9b2c9eb248ec None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:24.349331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321b8fc-55b3-4087-a3ac-02c13425b935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:24.349949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321b8fc-55b3-4087-a3ac-02c13425b935 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:24.350233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321b8fc-55b3-4087-a3ac-02c13425b935 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:24.350505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321b8fc-55b3-4087-a3ac-02c13425b935 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:24.351453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321b8fc-55b3-4087-a3ac-02c13425b935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:24.351758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321b8fc-55b3-4087-a3ac-02c13425b935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:24.353305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321b8fc-55b3-4087-a3ac-02c13425b935 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:24.365571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 635/1270] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:24.371424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a3701-16b8-407c-9aa8-092c9ce58f18 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:24.404495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:24.471338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a3701-16b8-407c-9aa8-092c9ce58f18 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:24.506676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86354c6d-f35c-49a8-a91a-431746097cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:24.507312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86354c6d-f35c-49a8-a91a-431746097cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:24.507597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86354c6d-f35c-49a8-a91a-431746097cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:24.507874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86354c6d-f35c-49a8-a91a-431746097cb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:24.515402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86354c6d-f35c-49a8-a91a-431746097cb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:24.516880 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86354c6d-f35c-49a8-a91a-431746097cb4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-86354c6d-f35c-49a8-a91a-431746097cb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:27:24.528825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:24.529904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:24.531172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:27:24.531172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:27:24.531172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:27:24.531172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:27:24.531172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:27:24.531172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:27:24.531172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:24.531172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:24.531172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 636/1271] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:27:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:24.535955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321b8fc-55b3-4087-a3ac-02c13425b935 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:24.594528 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:24.634466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321b8fc-55b3-4087-a3ac-02c13425b935 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:24.668746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28081c9e-03c1-42fe-8803-4c16287aa830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:24.669295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28081c9e-03c1-42fe-8803-4c16287aa830 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:24.669623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28081c9e-03c1-42fe-8803-4c16287aa830 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:24.669931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28081c9e-03c1-42fe-8803-4c16287aa830 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:24.670689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28081c9e-03c1-42fe-8803-4c16287aa830 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:24.671000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28081c9e-03c1-42fe-8803-4c16287aa830 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:24.672456 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28081c9e-03c1-42fe-8803-4c16287aa830 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:24.684336 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 636/1272] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:27:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:24.690554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86354c6d-f35c-49a8-a91a-431746097cb4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:24.779916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:24.788200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86354c6d-f35c-49a8-a91a-431746097cb4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:24.820910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c516b-8677-4c0f-82fb-6cee88227701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:24.821564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c516b-8677-4c0f-82fb-6cee88227701 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:24.821914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c516b-8677-4c0f-82fb-6cee88227701 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:24.822123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c516b-8677-4c0f-82fb-6cee88227701 None admin] PATH_INFO: `/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/affaeff3cd294584b7238909e0457f80/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:24.824215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c516b-8677-4c0f-82fb-6cee88227701 None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ed6718e0b46a69ba512b878ce710a, user_id=affaeff3cd294584b7238909e0457f80, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:24.833771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c516b-8677-4c0f-82fb-6cee88227701 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:24.952453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 637/1273] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Aug 4 07:27:24 2022] PUT /identity/v3/projects/a29ed6718e0b46a69ba512b878ce710a/users/affaeff3cd294584b7238909e0457f80/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:27:26.923915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481c903-afd0-4f00-a814-7fbf171b0ae1 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:26.924505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481c903-afd0-4f00-a814-7fbf171b0ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:26.924505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481c903-afd0-4f00-a814-7fbf171b0ae1 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:26.925819 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 637/1274] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:27:26.931134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd5233-2669-48d7-8169-b347536d4cdb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:26.931652 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd5233-2669-48d7-8169-b347536d4cdb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:26.931771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd5233-2669-48d7-8169-b347536d4cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:26.950475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6acd5233-2669-48d7-8169-b347536d4cdb None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:27:26.951671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6acd5233-2669-48d7-8169-b347536d4cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:27:27.019201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 638/1275] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:27:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:27:27.043203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481c903-afd0-4f00-a814-7fbf171b0ae1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:27.143040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481c903-afd0-4f00-a814-7fbf171b0ae1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:27.160285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:27.177204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce214c8-c19f-4c3d-97e7-f3cc1fafec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:27.177773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce214c8-c19f-4c3d-97e7-f3cc1fafec37 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:27.178081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce214c8-c19f-4c3d-97e7-f3cc1fafec37 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:27.178362 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce214c8-c19f-4c3d-97e7-f3cc1fafec37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:27.188163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce214c8-c19f-4c3d-97e7-f3cc1fafec37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8ce214c8-c19f-4c3d-97e7-f3cc1fafec37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 07:27:27.193159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:27.201695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:27.202617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 638/1276] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Aug 4 07:27:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:27.203384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd5233-2669-48d7-8169-b347536d4cdb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:27.280822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:27.298637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd5233-2669-48d7-8169-b347536d4cdb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:27.330566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ab487-f5e8-492e-a7fd-40c0c6f4ec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:27.331167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ab487-f5e8-492e-a7fd-40c0c6f4ec59 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:27.331450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ab487-f5e8-492e-a7fd-40c0c6f4ec59 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:27.331721 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ab487-f5e8-492e-a7fd-40c0c6f4ec59 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:27.332449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ab487-f5e8-492e-a7fd-40c0c6f4ec59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:27.332785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ab487-f5e8-492e-a7fd-40c0c6f4ec59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:27.334138 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ab487-f5e8-492e-a7fd-40c0c6f4ec59 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:27.342434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 639/1277] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Aug 4 07:27:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:27.348284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce214c8-c19f-4c3d-97e7-f3cc1fafec37 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:27.446856 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce214c8-c19f-4c3d-97e7-f3cc1fafec37 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:27.481367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd3493-7d03-465f-bf9d-22fdfa3ad755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:27.482000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd3493-7d03-465f-bf9d-22fdfa3ad755 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:27.482284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd3493-7d03-465f-bf9d-22fdfa3ad755 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:27.482559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd3493-7d03-465f-bf9d-22fdfa3ad755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:27.492313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd3493-7d03-465f-bf9d-22fdfa3ad755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:27.493752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd3493-7d03-465f-bf9d-22fdfa3ad755 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1abd3493-7d03-465f-bf9d-22fdfa3ad755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:27.508481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:27.509544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:27.510715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:27.510715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:27.510715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:27.510715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:27.510715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 639/1278] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:27.516129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ab487-f5e8-492e-a7fd-40c0c6f4ec59 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:27.614422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ab487-f5e8-492e-a7fd-40c0c6f4ec59 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:27.646860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5827b4-2dbc-47b4-b5d4-ebdf81773954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:27.647465 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5827b4-2dbc-47b4-b5d4-ebdf81773954 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:27.647750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5827b4-2dbc-47b4-b5d4-ebdf81773954 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:27.648048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5827b4-2dbc-47b4-b5d4-ebdf81773954 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:27.648729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5827b4-2dbc-47b4-b5d4-ebdf81773954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:27.649035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5827b4-2dbc-47b4-b5d4-ebdf81773954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:27.650404 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5827b4-2dbc-47b4-b5d4-ebdf81773954 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:27.662494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 640/1279] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:27.668296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd3493-7d03-465f-bf9d-22fdfa3ad755 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:27.698111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:27.767583 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd3493-7d03-465f-bf9d-22fdfa3ad755 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:27.799799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531fe52-c4e1-49d6-b16e-d154aa821123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:27.800459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531fe52-c4e1-49d6-b16e-d154aa821123 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:27.800745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531fe52-c4e1-49d6-b16e-d154aa821123 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:27.801045 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531fe52-c4e1-49d6-b16e-d154aa821123 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:27.809550 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531fe52-c4e1-49d6-b16e-d154aa821123 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:27.811092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531fe52-c4e1-49d6-b16e-d154aa821123 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6531fe52-c4e1-49d6-b16e-d154aa821123 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 07:27:27.822441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:27.823508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:27.825204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:27.825204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:27.825204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 07:27:27.825204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 07:27:27.825204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 07:27:27.825204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 07:27:27.825204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 07:27:27.825204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 07:27:27.825204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 07:27:27.825204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:27.825204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 640/1280] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:27.830191 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5827b4-2dbc-47b4-b5d4-ebdf81773954 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:27.835780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:27.927477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5827b4-2dbc-47b4-b5d4-ebdf81773954 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:27.960882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15703c-d06c-4cc0-b0eb-7373b5e76b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:27.966639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15703c-d06c-4cc0-b0eb-7373b5e76b76 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:27.966639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15703c-d06c-4cc0-b0eb-7373b5e76b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:27.966639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15703c-d06c-4cc0-b0eb-7373b5e76b76 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:27.966639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15703c-d06c-4cc0-b0eb-7373b5e76b76 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:27.966639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15703c-d06c-4cc0-b0eb-7373b5e76b76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:27.966639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15703c-d06c-4cc0-b0eb-7373b5e76b76 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:27.981935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 641/1281] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Aug 4 07:27:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:27.987999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531fe52-c4e1-49d6-b16e-d154aa821123 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:28.078282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:28.087749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531fe52-c4e1-49d6-b16e-d154aa821123 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:28.121852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3e1ef-17b6-4bc9-ac12-51369b93679d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:28.122474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3e1ef-17b6-4bc9-ac12-51369b93679d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:28.122759 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3e1ef-17b6-4bc9-ac12-51369b93679d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:28.123038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3e1ef-17b6-4bc9-ac12-51369b93679d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:28.190932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3e1ef-17b6-4bc9-ac12-51369b93679d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:28.192428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3e1ef-17b6-4bc9-ac12-51369b93679d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:28.279067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:28.288948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 641/1282] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Aug 4 07:27:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 07:27:28.294919 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15703c-d06c-4cc0-b0eb-7373b5e76b76 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:28.355308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:28.391901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15703c-d06c-4cc0-b0eb-7373b5e76b76 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:28.423436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e0294-1ae3-4530-bacf-390c824a395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:28.424079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e0294-1ae3-4530-bacf-390c824a395b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:28.424360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e0294-1ae3-4530-bacf-390c824a395b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:28.424630 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e0294-1ae3-4530-bacf-390c824a395b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:28.434771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e0294-1ae3-4530-bacf-390c824a395b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:28.436265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e0294-1ae3-4530-bacf-390c824a395b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-511e0294-1ae3-4530-bacf-390c824a395b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 07:27:28.450245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:28.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:28.452384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 07:27:28.452384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 07:27:28.452384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 07:27:28.452384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:28.452384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 642/1283] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:28.457143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3e1ef-17b6-4bc9-ac12-51369b93679d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:28.554782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3e1ef-17b6-4bc9-ac12-51369b93679d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:28.588709 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d546ff-5315-4476-90f1-92fa3708da06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:28.589345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d546ff-5315-4476-90f1-92fa3708da06 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:28.589693 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d546ff-5315-4476-90f1-92fa3708da06 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:28.589972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d546ff-5315-4476-90f1-92fa3708da06 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:28.590855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d546ff-5315-4476-90f1-92fa3708da06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:28.591254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d546ff-5315-4476-90f1-92fa3708da06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:28.592661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d546ff-5315-4476-90f1-92fa3708da06 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:28.604405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 642/1284] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Aug 4 07:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:28.610108 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e0294-1ae3-4530-bacf-390c824a395b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:28.710311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e0294-1ae3-4530-bacf-390c824a395b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:28.723421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:28.742573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff98ce3-078a-4f95-b1a1-e99175a75f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:28.743201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff98ce3-078a-4f95-b1a1-e99175a75f17 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:28.743485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff98ce3-078a-4f95-b1a1-e99175a75f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:28.743762 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff98ce3-078a-4f95-b1a1-e99175a75f17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:28.750727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff98ce3-078a-4f95-b1a1-e99175a75f17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:28.752195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff98ce3-078a-4f95-b1a1-e99175a75f17 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7ff98ce3-078a-4f95-b1a1-e99175a75f17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 07:27:28.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:28.765010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:28.766277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 07:27:28.766277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 07:27:28.766277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 07:27:28.766277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 07:27:28.766277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 07:27:28.766277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 07:27:28.766277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 07:27:28.766277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Aug 04 07:27:28.766277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 643/1285] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Aug 4 07:27:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:28.771061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d546ff-5315-4476-90f1-92fa3708da06 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:28.820643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:28.869614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d546ff-5315-4476-90f1-92fa3708da06 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:28.906130 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf205a34-9cb4-47f5-a4b4-de044c108edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:28.906766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf205a34-9cb4-47f5-a4b4-de044c108edd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:28.907049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf205a34-9cb4-47f5-a4b4-de044c108edd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:28.907325 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf205a34-9cb4-47f5-a4b4-de044c108edd None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:28.908134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf205a34-9cb4-47f5-a4b4-de044c108edd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:28.908444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf205a34-9cb4-47f5-a4b4-de044c108edd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:28.909937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf205a34-9cb4-47f5-a4b4-de044c108edd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:28.923091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 643/1286] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Aug 4 07:27:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:28.929355 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff98ce3-078a-4f95-b1a1-e99175a75f17 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:29.037228 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff98ce3-078a-4f95-b1a1-e99175a75f17 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:29.069551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4d5fc-7a7c-4b29-ab6d-a0ffe7d5f922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:29.070156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4d5fc-7a7c-4b29-ab6d-a0ffe7d5f922 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:29.070439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4d5fc-7a7c-4b29-ab6d-a0ffe7d5f922 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:29.070761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4d5fc-7a7c-4b29-ab6d-a0ffe7d5f922 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:29.071491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4d5fc-7a7c-4b29-ab6d-a0ffe7d5f922 None admin] RBAC: Adding query filter params (role.id=74ea89d30ac5491ab1d8fa1c7284be2e, scope.project.id=a29ed6718e0b46a69ba512b878ce710a, user.id=affaeff3cd294584b7238909e0457f80) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:29.071798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4d5fc-7a7c-4b29-ab6d-a0ffe7d5f922 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:29.073351 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4d5fc-7a7c-4b29-ab6d-a0ffe7d5f922 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:29.079725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 644/1287] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Aug 4 07:27:28 2022] GET /identity/v3/role_assignments?user.id=affaeff3cd294584b7238909e0457f80&scope.project.id=a29ed6718e0b46a69ba512b878ce710a&role.id=74ea89d30ac5491ab1d8fa1c7284be2e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:31.147222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e195929-af32-49b4-b8c4-21087f92e375 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:31.147745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e195929-af32-49b4-b8c4-21087f92e375 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:31.147854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e195929-af32-49b4-b8c4-21087f92e375 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:31.149170 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 644/1288] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:27:31.154805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653123c2-d573-4fad-9d58-9f9e373d93a6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:31.155213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653123c2-d573-4fad-9d58-9f9e373d93a6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:31.155355 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653123c2-d573-4fad-9d58-9f9e373d93a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:31.173499 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-653123c2-d573-4fad-9d58-9f9e373d93a6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:27:31.174682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-653123c2-d573-4fad-9d58-9f9e373d93a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:27:31.203590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:31.240282 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 645/1289] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:27:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:27:31.262362 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e195929-af32-49b4-b8c4-21087f92e375 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:31.362187 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e195929-af32-49b4-b8c4-21087f92e375 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:31.373436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:31.396497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369330f4-1aa7-4385-af33-4b46bed6c794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:31.397170 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369330f4-1aa7-4385-af33-4b46bed6c794 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:31.397454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369330f4-1aa7-4385-af33-4b46bed6c794 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:31.397725 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369330f4-1aa7-4385-af33-4b46bed6c794 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:31.398346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369330f4-1aa7-4385-af33-4b46bed6c794 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:31.399708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369330f4-1aa7-4385-af33-4b46bed6c794 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-369330f4-1aa7-4385-af33-4b46bed6c794 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:31.409044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:31.410219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:31.411075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 645/1290] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Aug 4 07:27:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:31.416660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653123c2-d573-4fad-9d58-9f9e373d93a6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:31.515106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653123c2-d573-4fad-9d58-9f9e373d93a6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:31.548617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a911101-3a6c-490c-82d1-bec23f7a9d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:31.549233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a911101-3a6c-490c-82d1-bec23f7a9d7f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:31.549515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a911101-3a6c-490c-82d1-bec23f7a9d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:31.549791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a911101-3a6c-490c-82d1-bec23f7a9d7f None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:31.550461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a911101-3a6c-490c-82d1-bec23f7a9d7f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:31.550764 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a911101-3a6c-490c-82d1-bec23f7a9d7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:31.552159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a911101-3a6c-490c-82d1-bec23f7a9d7f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:31.557008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 646/1291] 10.222.0.64 () {56 vars in 1116 bytes} [Thu Aug 4 07:27:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:31.563478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369330f4-1aa7-4385-af33-4b46bed6c794 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:31.660401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369330f4-1aa7-4385-af33-4b46bed6c794 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:31.692970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032b944-7c4a-4b6a-af2e-c1dbbeec6a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:31.693572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032b944-7c4a-4b6a-af2e-c1dbbeec6a13 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:31.693849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032b944-7c4a-4b6a-af2e-c1dbbeec6a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:31.694118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032b944-7c4a-4b6a-af2e-c1dbbeec6a13 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:31.694784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032b944-7c4a-4b6a-af2e-c1dbbeec6a13 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:31.695082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032b944-7c4a-4b6a-af2e-c1dbbeec6a13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:31.696459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032b944-7c4a-4b6a-af2e-c1dbbeec6a13 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:31.701444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 646/1292] 10.222.0.64 () {56 vars in 1116 bytes} [Thu Aug 4 07:27:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:33.707894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f335a9-7c6f-4b60-9587-287979d7099e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:33.708447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f335a9-7c6f-4b60-9587-287979d7099e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:33.708563 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f335a9-7c6f-4b60-9587-287979d7099e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:33.709970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 647/1293] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:27:33.716196 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8424c-d08b-4113-9b23-122312774237 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:33.721679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8424c-d08b-4113-9b23-122312774237 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:33.721679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8424c-d08b-4113-9b23-122312774237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:33.735247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-aba8424c-d08b-4113-9b23-122312774237 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:27:33.736471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aba8424c-d08b-4113-9b23-122312774237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:27:33.765978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:33.801986 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 647/1294] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:27:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:27:33.825763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f335a9-7c6f-4b60-9587-287979d7099e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:33.856410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:33.924837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f335a9-7c6f-4b60-9587-287979d7099e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:33.959893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc10e2-b76d-4468-8862-e974e5fba079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:33.960541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc10e2-b76d-4468-8862-e974e5fba079 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:33.960847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc10e2-b76d-4468-8862-e974e5fba079 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:33.961124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc10e2-b76d-4468-8862-e974e5fba079 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:33.961818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc10e2-b76d-4468-8862-e974e5fba079 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:33.963190 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc10e2-b76d-4468-8862-e974e5fba079 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:34.066280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 648/1295] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Aug 4 07:27:33 2022] POST /identity/v3/services => generated 246 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:27:36.103369 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbb186-6d3d-4e1c-9e3a-6ba8e980fa4c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:36.103908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbb186-6d3d-4e1c-9e3a-6ba8e980fa4c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:36.104077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbb186-6d3d-4e1c-9e3a-6ba8e980fa4c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:36.106065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 648/1296] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:27:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:27:36.111813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8e1de-65cd-4daa-bdaf-941dbf001b66 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:36.112296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8e1de-65cd-4daa-bdaf-941dbf001b66 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:36.112415 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8e1de-65cd-4daa-bdaf-941dbf001b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:36.130325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bde8e1de-65cd-4daa-bdaf-941dbf001b66 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:27:36.131481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bde8e1de-65cd-4daa-bdaf-941dbf001b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:27:36.201744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 649/1297] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:27:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:27:36.230973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbb186-6d3d-4e1c-9e3a-6ba8e980fa4c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:36.327641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbb186-6d3d-4e1c-9e3a-6ba8e980fa4c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:36.361477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006b940-7628-4724-9858-311bc1860ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:36.362103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006b940-7628-4724-9858-311bc1860ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:36.362382 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006b940-7628-4724-9858-311bc1860ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:36.362656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006b940-7628-4724-9858-311bc1860ad7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:36.363276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006b940-7628-4724-9858-311bc1860ad7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:36.364652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006b940-7628-4724-9858-311bc1860ad7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a006b940-7628-4724-9858-311bc1860ad7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:36.373235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:36.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:36.376359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 649/1298] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Aug 4 07:27:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:36.380525 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8e1de-65cd-4daa-bdaf-941dbf001b66 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:36.399443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:36.478724 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8e1de-65cd-4daa-bdaf-941dbf001b66 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:36.510856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453138d0-0345-40b0-81b6-51c4b2d249cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:36.511407 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453138d0-0345-40b0-81b6-51c4b2d249cd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:36.512344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453138d0-0345-40b0-81b6-51c4b2d249cd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:36.512523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453138d0-0345-40b0-81b6-51c4b2d249cd None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:36.513235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453138d0-0345-40b0-81b6-51c4b2d249cd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:36.513731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453138d0-0345-40b0-81b6-51c4b2d249cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:36.516049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453138d0-0345-40b0-81b6-51c4b2d249cd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:36.521167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 650/1299] 10.222.0.64 () {56 vars in 1116 bytes} [Thu Aug 4 07:27:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:36.527328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006b940-7628-4724-9858-311bc1860ad7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:36.558472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:36.624380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006b940-7628-4724-9858-311bc1860ad7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:36.658125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2313124-88a2-448c-a417-1eefc02806eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:36.658742 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2313124-88a2-448c-a417-1eefc02806eb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:36.659027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2313124-88a2-448c-a417-1eefc02806eb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:36.659303 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2313124-88a2-448c-a417-1eefc02806eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:36.660088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2313124-88a2-448c-a417-1eefc02806eb None admin] RBAC: Adding query filter params (interface=public, service_id=fad1a17d6427465bae537f12cb335ad1, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:36.660402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2313124-88a2-448c-a417-1eefc02806eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:36.661775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2313124-88a2-448c-a417-1eefc02806eb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:36.666713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 650/1300] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Aug 4 07:27:36 2022] GET /identity/v3/endpoints?service_id=fad1a17d6427465bae537f12cb335ad1&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:36.672620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453138d0-0345-40b0-81b6-51c4b2d249cd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:36.753524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:36.771475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453138d0-0345-40b0-81b6-51c4b2d249cd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:36.804545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad47437-8acb-4cff-9af4-9974ab45c39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:36.805173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad47437-8acb-4cff-9af4-9974ab45c39b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:36.805450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad47437-8acb-4cff-9af4-9974ab45c39b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:36.805719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad47437-8acb-4cff-9af4-9974ab45c39b None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:36.806397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad47437-8acb-4cff-9af4-9974ab45c39b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:36.807864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad47437-8acb-4cff-9af4-9974ab45c39b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:36.815090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 651/1301] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Aug 4 07:27:36 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:27:38.802437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba0061-f7b2-48ec-a614-edffd587945a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:38.803060 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba0061-f7b2-48ec-a614-edffd587945a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:38.803156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba0061-f7b2-48ec-a614-edffd587945a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:38.804509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 651/1302] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:27:38.809687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a537f3c-a2b9-479e-bcc6-a0f23337e496 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:38.810120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a537f3c-a2b9-479e-bcc6-a0f23337e496 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:38.810270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a537f3c-a2b9-479e-bcc6-a0f23337e496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:38.828705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2a537f3c-a2b9-479e-bcc6-a0f23337e496 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:27:38.829952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a537f3c-a2b9-479e-bcc6-a0f23337e496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:27:38.895377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 652/1303] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:27:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 07:27:38.918723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba0061-f7b2-48ec-a614-edffd587945a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:39.014689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:39.021253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba0061-f7b2-48ec-a614-edffd587945a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:39.054218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7626968-5e17-4387-94a0-c28dc1f74745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:39.054818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7626968-5e17-4387-94a0-c28dc1f74745 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:39.055104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7626968-5e17-4387-94a0-c28dc1f74745 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:39.055409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7626968-5e17-4387-94a0-c28dc1f74745 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:39.056067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7626968-5e17-4387-94a0-c28dc1f74745 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:39.057426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7626968-5e17-4387-94a0-c28dc1f74745 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c7626968-5e17-4387-94a0-c28dc1f74745 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 07:27:39.067306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 07:27:39.068536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Aug 04 07:27:39.069845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 652/1304] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Aug 4 07:27:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 07:27:39.075199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a537f3c-a2b9-479e-bcc6-a0f23337e496 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:39.149772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:39.170965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a537f3c-a2b9-479e-bcc6-a0f23337e496 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:39.204779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f0969-9ee9-414b-a913-ad603c4d9412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:39.205379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0969-9ee9-414b-a913-ad603c4d9412 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:39.205659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0969-9ee9-414b-a913-ad603c4d9412 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:39.205965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0969-9ee9-414b-a913-ad603c4d9412 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:39.206597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f0969-9ee9-414b-a913-ad603c4d9412 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:27:39.206894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f0969-9ee9-414b-a913-ad603c4d9412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:39.208274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f0969-9ee9-414b-a913-ad603c4d9412 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:39.213271 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 653/1305] 10.222.0.64 () {56 vars in 1116 bytes} [Thu Aug 4 07:27:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:27:39.219266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7626968-5e17-4387-94a0-c28dc1f74745 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:27:39.318901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7626968-5e17-4387-94a0-c28dc1f74745 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:27:39.352475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52addf-499c-4d4b-84b0-b518b864823d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:27:39.353082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52addf-499c-4d4b-84b0-b518b864823d None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:27:39.353367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52addf-499c-4d4b-84b0-b518b864823d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:27:39.353648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52addf-499c-4d4b-84b0-b518b864823d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:27:39.354361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52addf-499c-4d4b-84b0-b518b864823d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:27:39.356560 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52addf-499c-4d4b-84b0-b518b864823d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:27:39.437290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 653/1306] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:27:39 2022] POST /identity/v3/endpoints => generated 331 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:27:49.210293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=30 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:27:49.431943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:33:43.950852 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20621cb8-eb5e-4f4c-91eb-c08b2af84255 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:43.951433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20621cb8-eb5e-4f4c-91eb-c08b2af84255 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:43.951433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20621cb8-eb5e-4f4c-91eb-c08b2af84255 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:43.962081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b914d-86f0-422e-96f3-44b57bb6380d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:43.962621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b914d-86f0-422e-96f3-44b57bb6380d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:43.963035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b914d-86f0-422e-96f3-44b57bb6380d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:43.972321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 654/1307] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:33:43 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:33:44.065589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-be1b914d-86f0-422e-96f3-44b57bb6380d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:33:44.066992 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-be1b914d-86f0-422e-96f3-44b57bb6380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:44.210011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 654/1308] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:33:44.265491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f778a-e896-4291-bf6e-2e4d65d3883a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:44.265781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f778a-e896-4291-bf6e-2e4d65d3883a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:44.266061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f778a-e896-4291-bf6e-2e4d65d3883a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:44.267419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 655/1309] 10.222.0.64 () {54 vars in 965 bytes} [Thu Aug 4 07:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:33:44.279331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954baf3-02f3-476a-811c-bdb2533653d1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:44.280020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954baf3-02f3-476a-811c-bdb2533653d1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:44.280744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954baf3-02f3-476a-811c-bdb2533653d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:44.373765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a954baf3-02f3-476a-811c-bdb2533653d1 None None] MFA Rules not processed for user `b80e701a81cb4fa893e55288559010f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:33:44.375007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a954baf3-02f3-476a-811c-bdb2533653d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:44.397174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:33:44.467181 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f778a-e896-4291-bf6e-2e4d65d3883a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:33:44.468959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a78f778a-e896-4291-bf6e-2e4d65d3883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:44.473195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 655/1310] 10.222.0.64 () {58 vars in 1062 bytes} [Thu Aug 4 07:33:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:33:44.607434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f778a-e896-4291-bf6e-2e4d65d3883a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:33:44.689654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f606c8-08f3-45a7-86b8-52464011b0b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:33:44.690267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f606c8-08f3-45a7-86b8-52464011b0b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:44.690619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f606c8-08f3-45a7-86b8-52464011b0b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:44.691035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f606c8-08f3-45a7-86b8-52464011b0b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:44.692765 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-92f606c8-08f3-45a7-86b8-52464011b0b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:44.751076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:33:44.807208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f606c8-08f3-45a7-86b8-52464011b0b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:33:44.808759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f606c8-08f3-45a7-86b8-52464011b0b5 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:33:44.829167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 656/1311] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:33:47.160375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcab8b7-95e1-4994-9f07-e89fa1643584 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:47.160937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcab8b7-95e1-4994-9f07-e89fa1643584 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:47.160937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcab8b7-95e1-4994-9f07-e89fa1643584 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:47.162294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 656/1312] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:33:47.168154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab3431-3085-4e60-b106-a7b7af72a3c1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:47.168594 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab3431-3085-4e60-b106-a7b7af72a3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:47.168715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab3431-3085-4e60-b106-a7b7af72a3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:47.249970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d7ab3431-3085-4e60-b106-a7b7af72a3c1 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:33:47.251112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d7ab3431-3085-4e60-b106-a7b7af72a3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:47.354677 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 657/1313] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:33:47.393951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcab8b7-95e1-4994-9f07-e89fa1643584 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:33:47.395513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0fcab8b7-95e1-4994-9f07-e89fa1643584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:47.502292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcab8b7-95e1-4994-9f07-e89fa1643584 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:33:47.575680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:33:47.579220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dedce1-7842-445e-8f3c-801ba5c01b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:33:47.580183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dedce1-7842-445e-8f3c-801ba5c01b4e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:47.580472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dedce1-7842-445e-8f3c-801ba5c01b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:47.580751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dedce1-7842-445e-8f3c-801ba5c01b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:47.583153 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-73dedce1-7842-445e-8f3c-801ba5c01b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:47.703854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dedce1-7842-445e-8f3c-801ba5c01b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:33:47.705320 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dedce1-7842-445e-8f3c-801ba5c01b4e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:33:47.724878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 657/1314] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:33:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:33:50.292438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efcf50-c4af-439f-9d9c-3a8dbb33f901 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:50.293031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efcf50-c4af-439f-9d9c-3a8dbb33f901 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:50.293031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efcf50-c4af-439f-9d9c-3a8dbb33f901 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:50.294385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 658/1315] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:33:50.299426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc23e67-239b-4d0f-9ef2-a8949c5294e4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:50.299871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc23e67-239b-4d0f-9ef2-a8949c5294e4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:50.299977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc23e67-239b-4d0f-9ef2-a8949c5294e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:50.363677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dbc23e67-239b-4d0f-9ef2-a8949c5294e4 None None] MFA Rules not processed for user `60b0d29f45f6410f8f4e1856e03bc22b`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:33:50.364905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dbc23e67-239b-4d0f-9ef2-a8949c5294e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:50.415563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:33:50.459965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 658/1316] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:33:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:33:50.500480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efcf50-c4af-439f-9d9c-3a8dbb33f901 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:33:50.501996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a2efcf50-c4af-439f-9d9c-3a8dbb33f901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:50.512244 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:33:50.603017 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efcf50-c4af-439f-9d9c-3a8dbb33f901 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:33:50.673677 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdcf3c-c47b-4a8c-ba59-f7ba8672d6c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:33:50.674276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdcf3c-c47b-4a8c-ba59-f7ba8672d6c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:50.674552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdcf3c-c47b-4a8c-ba59-f7ba8672d6c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:50.674819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdcf3c-c47b-4a8c-ba59-f7ba8672d6c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:50.676428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5bcdcf3c-c47b-4a8c-ba59-f7ba8672d6c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:50.736706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:33:50.782063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdcf3c-c47b-4a8c-ba59-f7ba8672d6c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:33:50.783514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdcf3c-c47b-4a8c-ba59-f7ba8672d6c2 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:33:50.802247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 659/1317] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:33:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:33:53.630658 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec7064-c08b-4f20-a213-9cc3b4715bc4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:53.631129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec7064-c08b-4f20-a213-9cc3b4715bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:53.631238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec7064-c08b-4f20-a213-9cc3b4715bc4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:53.632581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 659/1318] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:33:53.638254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42291207-3b7c-47e1-b4cf-2d1bd61692f2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:53.638689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42291207-3b7c-47e1-b4cf-2d1bd61692f2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:53.638812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42291207-3b7c-47e1-b4cf-2d1bd61692f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:53.704433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-42291207-3b7c-47e1-b4cf-2d1bd61692f2 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:33:53.705635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-42291207-3b7c-47e1-b4cf-2d1bd61692f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:53.799973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 660/1319] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:33:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:33:53.839200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec7064-c08b-4f20-a213-9cc3b4715bc4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:33:53.840723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-feec7064-c08b-4f20-a213-9cc3b4715bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:53.942197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec7064-c08b-4f20-a213-9cc3b4715bc4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:33:54.015917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fbadd-a9d2-4892-afd5-45f54d233bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:33:54.016560 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fbadd-a9d2-4892-afd5-45f54d233bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:54.016847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fbadd-a9d2-4892-afd5-45f54d233bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:54.017123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fbadd-a9d2-4892-afd5-45f54d233bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:54.018748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4e6fbadd-a9d2-4892-afd5-45f54d233bb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:54.027054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:33:54.128356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fbadd-a9d2-4892-afd5-45f54d233bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:33:54.129723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fbadd-a9d2-4892-afd5-45f54d233bb0 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:33:54.149277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 660/1320] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:33:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:33:57.247836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999f0d1-fee4-4d25-937b-85d2beb91a1e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:57.248440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999f0d1-fee4-4d25-937b-85d2beb91a1e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:57.248440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999f0d1-fee4-4d25-937b-85d2beb91a1e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:57.249640 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 661/1321] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:33:57.254718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ed0e8-19f0-4674-bfb3-6c4ae7706d54 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:57.255104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ed0e8-19f0-4674-bfb3-6c4ae7706d54 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:57.255268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ed0e8-19f0-4674-bfb3-6c4ae7706d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:57.323007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-329ed0e8-19f0-4674-bfb3-6c4ae7706d54 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:33:57.324226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-329ed0e8-19f0-4674-bfb3-6c4ae7706d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:57.384470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:33:57.427851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 661/1322] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:33:57.479709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8640d-f398-42ca-b4c7-28daf3e4309d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:57.480007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8640d-f398-42ca-b4c7-28daf3e4309d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:57.480300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8640d-f398-42ca-b4c7-28daf3e4309d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:57.481696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 662/1323] 10.222.0.64 () {54 vars in 965 bytes} [Thu Aug 4 07:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:33:57.488237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ff4f1-d926-4723-a945-203b5e11bab6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:57.488530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ff4f1-d926-4723-a945-203b5e11bab6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:57.488809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ff4f1-d926-4723-a945-203b5e11bab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:57.571516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-620ff4f1-d926-4723-a945-203b5e11bab6 None None] MFA Rules not processed for user `b80e701a81cb4fa893e55288559010f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:33:57.572784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-620ff4f1-d926-4723-a945-203b5e11bab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:57.650245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 662/1324] 10.222.0.64 () {58 vars in 1062 bytes} [Thu Aug 4 07:33:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:33:57.659038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8640d-f398-42ca-b4c7-28daf3e4309d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:33:57.660657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bcd8640d-f398-42ca-b4c7-28daf3e4309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:57.680715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:33:57.762876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8640d-f398-42ca-b4c7-28daf3e4309d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:33:57.839076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5fb64-1e46-4299-a2de-08372ab5f7f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:33:57.839583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5fb64-1e46-4299-a2de-08372ab5f7f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:33:57.839865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5fb64-1e46-4299-a2de-08372ab5f7f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:33:57.840176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5fb64-1e46-4299-a2de-08372ab5f7f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:33:57.841809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98e5fb64-1e46-4299-a2de-08372ab5f7f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:33:57.913357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:33:57.952921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5fb64-1e46-4299-a2de-08372ab5f7f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:33:57.954399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5fb64-1e46-4299-a2de-08372ab5f7f2 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:33:57.974615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 663/1325] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:33:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:00.307734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51672a-8aca-49d4-8dbd-21a70b3f1668 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:00.308330 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51672a-8aca-49d4-8dbd-21a70b3f1668 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:00.308330 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51672a-8aca-49d4-8dbd-21a70b3f1668 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:00.309739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 663/1326] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:00.314767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c35d50-d6f3-4e9c-a168-a0abf6702062 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:00.315198 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c35d50-d6f3-4e9c-a168-a0abf6702062 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:00.315332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c35d50-d6f3-4e9c-a168-a0abf6702062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:00.383590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-63c35d50-d6f3-4e9c-a168-a0abf6702062 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:00.384821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-63c35d50-d6f3-4e9c-a168-a0abf6702062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:00.603549 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 664/1327] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:00.640668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51672a-8aca-49d4-8dbd-21a70b3f1668 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:00.642209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5d51672a-8aca-49d4-8dbd-21a70b3f1668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:00.702955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:00.747198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51672a-8aca-49d4-8dbd-21a70b3f1668 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:00.818972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28494cd7-5130-404f-aee8-6ce50aca7f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:00.819586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28494cd7-5130-404f-aee8-6ce50aca7f09 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:00.819863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28494cd7-5130-404f-aee8-6ce50aca7f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:00.820245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28494cd7-5130-404f-aee8-6ce50aca7f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:00.821865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-28494cd7-5130-404f-aee8-6ce50aca7f09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:00.925818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:00.928542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28494cd7-5130-404f-aee8-6ce50aca7f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:00.930867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28494cd7-5130-404f-aee8-6ce50aca7f09 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:00.950114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 664/1328] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:03.039884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4364a-2e80-487b-bf57-d8636ea579a9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:03.040467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4364a-2e80-487b-bf57-d8636ea579a9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:03.040467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4364a-2e80-487b-bf57-d8636ea579a9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:03.041803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 665/1329] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:03.046897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef1ca7-4254-48a3-93cc-5298bc7fb97b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:03.047330 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef1ca7-4254-48a3-93cc-5298bc7fb97b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:03.047498 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef1ca7-4254-48a3-93cc-5298bc7fb97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:03.125971 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fdef1ca7-4254-48a3-93cc-5298bc7fb97b None None] MFA Rules not processed for user `60b0d29f45f6410f8f4e1856e03bc22b`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:03.127787 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fdef1ca7-4254-48a3-93cc-5298bc7fb97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:03.233804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 665/1330] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:03.271712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4364a-2e80-487b-bf57-d8636ea579a9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:03.274058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2da4364a-2e80-487b-bf57-d8636ea579a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:03.303963 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:03.376580 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4364a-2e80-487b-bf57-d8636ea579a9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:03.449046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca48f8-6b75-432c-b2e8-cfe533c1528b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:03.449593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca48f8-6b75-432c-b2e8-cfe533c1528b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:03.449875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca48f8-6b75-432c-b2e8-cfe533c1528b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:03.450147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca48f8-6b75-432c-b2e8-cfe533c1528b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:03.451757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-72ca48f8-6b75-432c-b2e8-cfe533c1528b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:03.527562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:03.559466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca48f8-6b75-432c-b2e8-cfe533c1528b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:03.560948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca48f8-6b75-432c-b2e8-cfe533c1528b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:03.579204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 666/1331] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:03 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:06.406754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e29103-f9e8-4da9-b902-b35f2c6296f1 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:06.407286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e29103-f9e8-4da9-b902-b35f2c6296f1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:06.407286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e29103-f9e8-4da9-b902-b35f2c6296f1 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:06.408611 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 666/1332] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:06.413935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908a9df-99ba-4232-aaaa-96d322d1554f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:06.414381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908a9df-99ba-4232-aaaa-96d322d1554f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:06.414489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908a9df-99ba-4232-aaaa-96d322d1554f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:06.480985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b908a9df-99ba-4232-aaaa-96d322d1554f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:06.482185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b908a9df-99ba-4232-aaaa-96d322d1554f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:06.576559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 667/1333] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:06.613188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e29103-f9e8-4da9-b902-b35f2c6296f1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:06.614728 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c5e29103-f9e8-4da9-b902-b35f2c6296f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:06.684818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:06.717112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e29103-f9e8-4da9-b902-b35f2c6296f1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:06.789161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc45ce-cb6f-41f1-8618-163e99b4670f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:06.789817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc45ce-cb6f-41f1-8618-163e99b4670f service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:06.790104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc45ce-cb6f-41f1-8618-163e99b4670f service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:06.790386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc45ce-cb6f-41f1-8618-163e99b4670f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:06.792091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a8fc45ce-cb6f-41f1-8618-163e99b4670f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:06.898053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc45ce-cb6f-41f1-8618-163e99b4670f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:06.899487 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc45ce-cb6f-41f1-8618-163e99b4670f service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:06.909460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:06.917780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 667/1334] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:08.977270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a33a-5175-4924-b2d7-36cf2bd4b74e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:08.977727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a33a-5175-4924-b2d7-36cf2bd4b74e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:08.977833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a33a-5175-4924-b2d7-36cf2bd4b74e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:08.979174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 668/1335] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:08.984315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b40cb-66ba-4d55-8e3c-85d96296fa41 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:08.988389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b40cb-66ba-4d55-8e3c-85d96296fa41 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:08.988389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b40cb-66ba-4d55-8e3c-85d96296fa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:09.049147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3b7b40cb-66ba-4d55-8e3c-85d96296fa41 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:09.050340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3b7b40cb-66ba-4d55-8e3c-85d96296fa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:09.147419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 668/1336] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:09.190798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a33a-5175-4924-b2d7-36cf2bd4b74e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:09.192350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6076a33a-5175-4924-b2d7-36cf2bd4b74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:09.231832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:09.294405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a33a-5175-4924-b2d7-36cf2bd4b74e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:09.364791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc883e0-3ce7-4e9e-a9fc-d7774ea77fd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:09.365399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc883e0-3ce7-4e9e-a9fc-d7774ea77fd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:09.365679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc883e0-3ce7-4e9e-a9fc-d7774ea77fd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:09.365954 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc883e0-3ce7-4e9e-a9fc-d7774ea77fd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:09.367684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5cc883e0-3ce7-4e9e-a9fc-d7774ea77fd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:09.446836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:09.471519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc883e0-3ce7-4e9e-a9fc-d7774ea77fd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:09.473019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc883e0-3ce7-4e9e-a9fc-d7774ea77fd7 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:09.492140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 669/1337] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:12.185697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568d929-15d0-45ca-b36d-807077f10e6f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:12.186224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568d929-15d0-45ca-b36d-807077f10e6f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:12.186315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568d929-15d0-45ca-b36d-807077f10e6f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:12.187668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 669/1338] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:12.192970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788866ce-6aa4-4b5c-bd23-0c06e005f264 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:12.193356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788866ce-6aa4-4b5c-bd23-0c06e005f264 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:12.193553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788866ce-6aa4-4b5c-bd23-0c06e005f264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:12.260209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-788866ce-6aa4-4b5c-bd23-0c06e005f264 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:12.261337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-788866ce-6aa4-4b5c-bd23-0c06e005f264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:12.356248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 670/1339] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:12.394099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568d929-15d0-45ca-b36d-807077f10e6f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:12.395616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e568d929-15d0-45ca-b36d-807077f10e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:12.470035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:12.498113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568d929-15d0-45ca-b36d-807077f10e6f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:12.571244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba5cb2-e770-4cd5-a48d-cc2f0044e338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:12.571782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba5cb2-e770-4cd5-a48d-cc2f0044e338 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:12.572101 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba5cb2-e770-4cd5-a48d-cc2f0044e338 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:12.572384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba5cb2-e770-4cd5-a48d-cc2f0044e338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:12.574017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0cba5cb2-e770-4cd5-a48d-cc2f0044e338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:12.680379 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba5cb2-e770-4cd5-a48d-cc2f0044e338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:12.681802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba5cb2-e770-4cd5-a48d-cc2f0044e338 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:12.699804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 670/1340] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:16.969706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b177113-4dad-4fab-a47a-4dbf9d09f348 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:16.970176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b177113-4dad-4fab-a47a-4dbf9d09f348 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:16.970267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b177113-4dad-4fab-a47a-4dbf9d09f348 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:16.971647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 671/1341] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:16.977450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0c839-962b-4979-a2e4-5a71664dd5f3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:16.977863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0c839-962b-4979-a2e4-5a71664dd5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:16.977998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0c839-962b-4979-a2e4-5a71664dd5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:16.983429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:17.046634 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3bb0c839-962b-4979-a2e4-5a71664dd5f3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:17.047844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3bb0c839-962b-4979-a2e4-5a71664dd5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:17.146540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 671/1342] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:34:17.186525 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b177113-4dad-4fab-a47a-4dbf9d09f348 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:17.188086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7b177113-4dad-4fab-a47a-4dbf9d09f348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:17.236983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:17.292758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b177113-4dad-4fab-a47a-4dbf9d09f348 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:17.366089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094fc07-498f-43e0-9058-61e646a92860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:17.367172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094fc07-498f-43e0-9058-61e646a92860 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:17.367455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094fc07-498f-43e0-9058-61e646a92860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:17.367731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094fc07-498f-43e0-9058-61e646a92860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:17.369387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2094fc07-498f-43e0-9058-61e646a92860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:17.460923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:17.477565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094fc07-498f-43e0-9058-61e646a92860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:17.479069 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094fc07-498f-43e0-9058-61e646a92860 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:17.498611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 672/1343] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:20.404551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62307-2538-4a82-a5b3-5343925e0f83 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:20.405064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62307-2538-4a82-a5b3-5343925e0f83 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:20.405181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62307-2538-4a82-a5b3-5343925e0f83 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:20.406467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 672/1344] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:20.412319 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147cb08-4686-4caa-a09c-692993462f98 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:20.412893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147cb08-4686-4caa-a09c-692993462f98 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:20.412893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147cb08-4686-4caa-a09c-692993462f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:20.479543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e147cb08-4686-4caa-a09c-692993462f98 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:20.480784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e147cb08-4686-4caa-a09c-692993462f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:20.577045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 673/1345] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:20.615555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e62307-2538-4a82-a5b3-5343925e0f83 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:20.617164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-57e62307-2538-4a82-a5b3-5343925e0f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:20.702766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:20.720134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e62307-2538-4a82-a5b3-5343925e0f83 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:20.795477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a2da7-b452-44ee-adcd-c0e3a7480660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:20.796136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a2da7-b452-44ee-adcd-c0e3a7480660 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:20.796423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a2da7-b452-44ee-adcd-c0e3a7480660 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:20.796700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a2da7-b452-44ee-adcd-c0e3a7480660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:20.798346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-808a2da7-b452-44ee-adcd-c0e3a7480660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:20.906165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a2da7-b452-44ee-adcd-c0e3a7480660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:20.907611 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a2da7-b452-44ee-adcd-c0e3a7480660 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:20.925998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 673/1346] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:25.120206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8695e3a-7fa8-43c1-89a1-c3d1b6365e70 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:25.120732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8695e3a-7fa8-43c1-89a1-c3d1b6365e70 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:25.120844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8695e3a-7fa8-43c1-89a1-c3d1b6365e70 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:25.122208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 674/1347] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:25.127689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd755e-a821-46cd-89a3-7804ba03c333 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:25.128193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd755e-a821-46cd-89a3-7804ba03c333 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:25.128305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd755e-a821-46cd-89a3-7804ba03c333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:25.144055 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:25.198058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c2dd755e-a821-46cd-89a3-7804ba03c333 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:25.199384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c2dd755e-a821-46cd-89a3-7804ba03c333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:25.321102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 674/1348] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:25.360800 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8695e3a-7fa8-43c1-89a1-c3d1b6365e70 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:25.362447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b8695e3a-7fa8-43c1-89a1-c3d1b6365e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:25.424592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:25.473477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8695e3a-7fa8-43c1-89a1-c3d1b6365e70 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:25.548443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014140ff-b50a-4f32-85a6-bceb35b30231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:25.549065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014140ff-b50a-4f32-85a6-bceb35b30231 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:25.549348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014140ff-b50a-4f32-85a6-bceb35b30231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:25.549615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014140ff-b50a-4f32-85a6-bceb35b30231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:25.551255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-014140ff-b50a-4f32-85a6-bceb35b30231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:25.679072 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:25.683040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014140ff-b50a-4f32-85a6-bceb35b30231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:25.684552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014140ff-b50a-4f32-85a6-bceb35b30231 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:25.706582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 675/1349] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:30.665339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf89844-e4e8-4f17-953b-57a3b6284f75 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:30.666349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf89844-e4e8-4f17-953b-57a3b6284f75 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:30.666819 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf89844-e4e8-4f17-953b-57a3b6284f75 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:30.668629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 675/1350] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:34:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:30.674032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9f79e-7375-43b9-a50d-33baf226fad5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:30.681224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9f79e-7375-43b9-a50d-33baf226fad5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:30.681224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9f79e-7375-43b9-a50d-33baf226fad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:30.763270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d5d9f79e-7375-43b9-a50d-33baf226fad5 None None] MFA Rules not processed for user `affaeff3cd294584b7238909e0457f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:30.764421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5d9f79e-7375-43b9-a50d-33baf226fad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:30.925076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 676/1351] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:34:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:35.166367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2e5c1-2d49-47e5-bb3c-cfb65590bdbf None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:35.166907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2e5c1-2d49-47e5-bb3c-cfb65590bdbf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:35.166907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2e5c1-2d49-47e5-bb3c-cfb65590bdbf None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:35.168485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 676/1352] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:35.173527 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3af8-4d3f-40dc-bbc3-79d315a8b05f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:35.174012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3af8-4d3f-40dc-bbc3-79d315a8b05f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:35.174309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3af8-4d3f-40dc-bbc3-79d315a8b05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:35.230532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:35.253692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2b0b3af8-4d3f-40dc-bbc3-79d315a8b05f None None] MFA Rules not processed for user `affaeff3cd294584b7238909e0457f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:35.255030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2b0b3af8-4d3f-40dc-bbc3-79d315a8b05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:35.318485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=59 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:35.369618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 677/1353] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:34:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:36.720302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385e92-371f-402e-b56f-d60706549bcd None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:36.720858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385e92-371f-402e-b56f-d60706549bcd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:36.720858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385e92-371f-402e-b56f-d60706549bcd None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:36.722210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 677/1354] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:36.734831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3bfa9-166a-455e-a170-8ece09edfbad None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:36.735274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3bfa9-166a-455e-a170-8ece09edfbad None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:36.735427 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3bfa9-166a-455e-a170-8ece09edfbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:36.812823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-abe3bfa9-166a-455e-a170-8ece09edfbad None None] MFA Rules not processed for user `affaeff3cd294584b7238909e0457f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:36.814026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-abe3bfa9-166a-455e-a170-8ece09edfbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:36.857154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:36.923852 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 678/1355] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:34:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:34:40.299313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6892c20-ade5-430d-ab0f-21970fa3c1d1 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:40.300215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6892c20-ade5-430d-ab0f-21970fa3c1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:40.300649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6892c20-ade5-430d-ab0f-21970fa3c1d1 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:40.303640 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 678/1356] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:40.307806 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251384da-61f4-433d-8830-a9113b818dc6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:40.308385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251384da-61f4-433d-8830-a9113b818dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:40.308385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251384da-61f4-433d-8830-a9113b818dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:40.387038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-251384da-61f4-433d-8830-a9113b818dc6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:40.388183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-251384da-61f4-433d-8830-a9113b818dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:40.493157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 679/1357] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:40.547628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b2253-8ee1-4bbc-9199-b3a7c401a2c8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:40.548420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b2253-8ee1-4bbc-9199-b3a7c401a2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:40.548934 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b2253-8ee1-4bbc-9199-b3a7c401a2c8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:40.550448 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 679/1358] 10.222.0.64 () {54 vars in 962 bytes} [Thu Aug 4 07:34:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:40.560530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d9679-3d73-435f-8017-09a843ed8d9d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:40.561266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d9679-3d73-435f-8017-09a843ed8d9d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:40.561713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d9679-3d73-435f-8017-09a843ed8d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:40.580253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:40.650143 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9a0d9679-3d73-435f-8017-09a843ed8d9d None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:40.651475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9a0d9679-3d73-435f-8017-09a843ed8d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:40.794777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 680/1359] 10.222.0.64 () {58 vars in 1059 bytes} [Thu Aug 4 07:34:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:40.803574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b2253-8ee1-4bbc-9199-b3a7c401a2c8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:40.805372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-554b2253-8ee1-4bbc-9199-b3a7c401a2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:40.957794 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b2253-8ee1-4bbc-9199-b3a7c401a2c8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:41.042741 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:41.090003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ce0db-e38c-4fc5-9748-7bf3fa0c7fc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:41.090730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ce0db-e38c-4fc5-9748-7bf3fa0c7fc6 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:41.091094 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ce0db-e38c-4fc5-9748-7bf3fa0c7fc6 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:41.091449 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ce0db-e38c-4fc5-9748-7bf3fa0c7fc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:41.093229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f23ce0db-e38c-4fc5-9748-7bf3fa0c7fc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:41.196066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ce0db-e38c-4fc5-9748-7bf3fa0c7fc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:41.197802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ce0db-e38c-4fc5-9748-7bf3fa0c7fc6 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:41.217509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 680/1360] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:43.467650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8829a-3158-41e4-99ee-351153646b4d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:43.468189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8829a-3158-41e4-99ee-351153646b4d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:43.468189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8829a-3158-41e4-99ee-351153646b4d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:43.470165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 681/1361] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:43.475625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33223-7481-480e-864c-4925f336f85d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:43.476204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33223-7481-480e-864c-4925f336f85d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:43.476204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33223-7481-480e-864c-4925f336f85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:43.547035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e5f33223-7481-480e-864c-4925f336f85d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:43.548333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e5f33223-7481-480e-864c-4925f336f85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:43.588059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:43.657074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 681/1362] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:43.701746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8829a-3158-41e4-99ee-351153646b4d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:43.703320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-94f8829a-3158-41e4-99ee-351153646b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:43.782692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:43.882043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8829a-3158-41e4-99ee-351153646b4d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:44.038296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6097854-00bc-4f5a-b69c-3d4afc8cfaba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:44.038805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6097854-00bc-4f5a-b69c-3d4afc8cfaba service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:44.039090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6097854-00bc-4f5a-b69c-3d4afc8cfaba service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:44.043945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6097854-00bc-4f5a-b69c-3d4afc8cfaba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:44.043945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e6097854-00bc-4f5a-b69c-3d4afc8cfaba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:44.066442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:44.140629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6097854-00bc-4f5a-b69c-3d4afc8cfaba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:44.142221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6097854-00bc-4f5a-b69c-3d4afc8cfaba service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:44.164332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 682/1363] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:46.592568 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36759f7-cea8-482e-9ff2-089a28776d7d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:46.593430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36759f7-cea8-482e-9ff2-089a28776d7d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:46.593958 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36759f7-cea8-482e-9ff2-089a28776d7d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:46.595461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 682/1364] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:46.601931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c80c7b-4906-454b-9afc-67fa45dc2822 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:46.602458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c80c7b-4906-454b-9afc-67fa45dc2822 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:46.602566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c80c7b-4906-454b-9afc-67fa45dc2822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:46.684234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-99c80c7b-4906-454b-9afc-67fa45dc2822 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:46.685453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-99c80c7b-4906-454b-9afc-67fa45dc2822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:46.784920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:46.791777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 683/1365] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:46.833767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5928205-6f7b-4830-aa28-b98cb21456c9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:46.834660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5928205-6f7b-4830-aa28-b98cb21456c9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:46.835081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5928205-6f7b-4830-aa28-b98cb21456c9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:46.836501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 683/1366] 10.222.0.64 () {54 vars in 962 bytes} [Thu Aug 4 07:34:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:46.844350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d39a30-f35f-4d98-9bf5-0fa1e430f1e5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:46.844578 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d39a30-f35f-4d98-9bf5-0fa1e430f1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:46.844861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d39a30-f35f-4d98-9bf5-0fa1e430f1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:46.928611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-22d39a30-f35f-4d98-9bf5-0fa1e430f1e5 None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:46.929885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-22d39a30-f35f-4d98-9bf5-0fa1e430f1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:47.065016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 684/1367] 10.222.0.64 () {58 vars in 1059 bytes} [Thu Aug 4 07:34:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:47.076411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5928205-6f7b-4830-aa28-b98cb21456c9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:47.078272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e5928205-6f7b-4830-aa28-b98cb21456c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:47.241907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5928205-6f7b-4830-aa28-b98cb21456c9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:47.263291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:47.363179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afa3c0-943d-4574-ac73-05a7d076d855 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:47.363959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afa3c0-943d-4574-ac73-05a7d076d855 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:47.364370 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afa3c0-943d-4574-ac73-05a7d076d855 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:47.364794 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afa3c0-943d-4574-ac73-05a7d076d855 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:47.367267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e2afa3c0-943d-4574-ac73-05a7d076d855 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:47.465082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afa3c0-943d-4574-ac73-05a7d076d855 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:47.467924 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afa3c0-943d-4574-ac73-05a7d076d855 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:47.488381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 684/1368] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:48.470452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797edad-2b83-4b85-848b-b7bd7caab75f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:48.471031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797edad-2b83-4b85-848b-b7bd7caab75f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:48.471031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797edad-2b83-4b85-848b-b7bd7caab75f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:48.472529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 685/1369] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 07:34:48.478128 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c68f2-8b9a-49e6-a769-9610d0fa1fd3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:48.478570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c68f2-8b9a-49e6-a769-9610d0fa1fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:48.478754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c68f2-8b9a-49e6-a769-9610d0fa1fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:48.513718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:48.551720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a45c68f2-8b9a-49e6-a769-9610d0fa1fd3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:48.554553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a45c68f2-8b9a-49e6-a769-9610d0fa1fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:48.656364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 685/1370] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:49.009745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d26d7-25d8-4682-a4d9-d972e75afe22 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:49.010798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d26d7-25d8-4682-a4d9-d972e75afe22 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:49.010798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d26d7-25d8-4682-a4d9-d972e75afe22 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:49.012512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 686/1371] 10.222.0.64 () {54 vars in 964 bytes} [Thu Aug 4 07:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:49.017392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197babe-81a0-4eee-bd39-a51b6b30d5c8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:49.017689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197babe-81a0-4eee-bd39-a51b6b30d5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:49.017970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197babe-81a0-4eee-bd39-a51b6b30d5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:49.101880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8197babe-81a0-4eee-bd39-a51b6b30d5c8 None None] MFA Rules not processed for user `eab096d4ccae4bc09e6177afd85a55de`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:49.103191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8197babe-81a0-4eee-bd39-a51b6b30d5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:49.150775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:49.183140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 686/1372] 10.222.0.64 () {58 vars in 1061 bytes} [Thu Aug 4 07:34:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:49.190796 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d26d7-25d8-4682-a4d9-d972e75afe22 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:49.192377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-951d26d7-25d8-4682-a4d9-d972e75afe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:49.248283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:49.305735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d26d7-25d8-4682-a4d9-d972e75afe22 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:49.383664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720a37d-c26a-416e-a189-660322d96f8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:49.384971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720a37d-c26a-416e-a189-660322d96f8a service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:49.385324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720a37d-c26a-416e-a189-660322d96f8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:49.385664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720a37d-c26a-416e-a189-660322d96f8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:49.387358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f720a37d-c26a-416e-a189-660322d96f8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:49.498548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:49.512679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720a37d-c26a-416e-a189-660322d96f8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:49.514146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720a37d-c26a-416e-a189-660322d96f8a service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:49.533061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 687/1373] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:49.787859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e1979-526e-4113-bc7c-3cbfc0eea0c8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:49.787859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e1979-526e-4113-bc7c-3cbfc0eea0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:49.788983 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e1979-526e-4113-bc7c-3cbfc0eea0c8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:49.796673 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 687/1374] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:49.796774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93257ea9-96d9-4bc9-829a-247b45c653d9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:49.796774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93257ea9-96d9-4bc9-829a-247b45c653d9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:49.797047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93257ea9-96d9-4bc9-829a-247b45c653d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:49.870563 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-93257ea9-96d9-4bc9-829a-247b45c653d9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:49.871781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-93257ea9-96d9-4bc9-829a-247b45c653d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:49.999545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 688/1375] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:50.062187 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e1979-526e-4113-bc7c-3cbfc0eea0c8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:50.066677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-041e1979-526e-4113-bc7c-3cbfc0eea0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:50.248244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e1979-526e-4113-bc7c-3cbfc0eea0c8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:50.305603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:50.380364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b80d7f-89a5-4c3a-ba09-143a4fe67a4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:50.382753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b80d7f-89a5-4c3a-ba09-143a4fe67a4d service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:50.383045 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b80d7f-89a5-4c3a-ba09-143a4fe67a4d service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:50.383322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b80d7f-89a5-4c3a-ba09-143a4fe67a4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:50.385625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-57b80d7f-89a5-4c3a-ba09-143a4fe67a4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:50.488693 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b80d7f-89a5-4c3a-ba09-143a4fe67a4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:50.490522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b80d7f-89a5-4c3a-ba09-143a4fe67a4d service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:50.509545 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 688/1376] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:52.667272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fee84-79e9-4c9d-846d-dc3d1c20502d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:52.667826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fee84-79e9-4c9d-846d-dc3d1c20502d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:52.667826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fee84-79e9-4c9d-846d-dc3d1c20502d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:52.669377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 689/1377] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:52.674313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64e3d3-fc56-4783-a41e-dcb916a8ee28 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:52.675109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64e3d3-fc56-4783-a41e-dcb916a8ee28 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:52.675626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64e3d3-fc56-4783-a41e-dcb916a8ee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:52.747556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-da64e3d3-fc56-4783-a41e-dcb916a8ee28 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:52.748838 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-da64e3d3-fc56-4783-a41e-dcb916a8ee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:52.755426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:52.858236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 689/1378] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:52.902266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fee84-79e9-4c9d-846d-dc3d1c20502d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:52.903862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-602fee84-79e9-4c9d-846d-dc3d1c20502d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:52.966858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:53.054129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fee84-79e9-4c9d-846d-dc3d1c20502d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:53.183335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a00694-8482-4440-a056-31efada12efd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:53.183952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00694-8482-4440-a056-31efada12efd service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:53.184265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00694-8482-4440-a056-31efada12efd service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:53.184636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00694-8482-4440-a056-31efada12efd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:53.186239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-88a00694-8482-4440-a056-31efada12efd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:53.211093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:53.324924 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a00694-8482-4440-a056-31efada12efd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:53.326485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a00694-8482-4440-a056-31efada12efd service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:53.352844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 690/1379] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:55.480125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ffa42-d7db-4ff0-9194-f46ee36280dc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:55.480711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ffa42-d7db-4ff0-9194-f46ee36280dc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:55.480711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ffa42-d7db-4ff0-9194-f46ee36280dc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:55.482180 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 690/1380] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:55.487122 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0df0f6-20da-4b4d-99b7-d0cfa5a6678f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:55.487508 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0df0f6-20da-4b4d-99b7-d0cfa5a6678f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:55.487670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0df0f6-20da-4b4d-99b7-d0cfa5a6678f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:55.558132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ab0df0f6-20da-4b4d-99b7-d0cfa5a6678f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:55.559313 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ab0df0f6-20da-4b4d-99b7-d0cfa5a6678f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:55.660523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:55.666910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 691/1381] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:55.711825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ffa42-d7db-4ff0-9194-f46ee36280dc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:55.713489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-928ffa42-d7db-4ff0-9194-f46ee36280dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:55.865358 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:55.878462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ffa42-d7db-4ff0-9194-f46ee36280dc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:56.005272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aae7d9-8621-4eb5-8167-3dfa6be6a222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:56.006043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aae7d9-8621-4eb5-8167-3dfa6be6a222 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:56.006432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aae7d9-8621-4eb5-8167-3dfa6be6a222 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:56.006808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aae7d9-8621-4eb5-8167-3dfa6be6a222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:56.008798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-05aae7d9-8621-4eb5-8167-3dfa6be6a222 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:56.124588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aae7d9-8621-4eb5-8167-3dfa6be6a222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:56.126272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aae7d9-8621-4eb5-8167-3dfa6be6a222 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:56.139129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:56.148477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 691/1382] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:34:58.614964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803d64d-6ed9-4822-b891-89a40c479f61 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:58.615536 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803d64d-6ed9-4822-b891-89a40c479f61 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:58.615699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803d64d-6ed9-4822-b891-89a40c479f61 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:58.617555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 692/1383] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:34:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:34:58.622852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5a14d-7429-4eab-b2ce-8f3dabff8a42 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:58.623359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5a14d-7429-4eab-b2ce-8f3dabff8a42 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:58.623529 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5a14d-7429-4eab-b2ce-8f3dabff8a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:58.696514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b5b5a14d-7429-4eab-b2ce-8f3dabff8a42 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:34:58.698281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b5b5a14d-7429-4eab-b2ce-8f3dabff8a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:58.796473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 692/1384] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:34:58.840361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803d64d-6ed9-4822-b891-89a40c479f61 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:34:58.841948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c803d64d-6ed9-4822-b891-89a40c479f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:59.010595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803d64d-6ed9-4822-b891-89a40c479f61 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:34:59.088158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:34:59.137569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1939693-2308-481d-9bfa-ffd1a8b34b47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:34:59.138202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1939693-2308-481d-9bfa-ffd1a8b34b47 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:34:59.138487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1939693-2308-481d-9bfa-ffd1a8b34b47 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:34:59.138764 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1939693-2308-481d-9bfa-ffd1a8b34b47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:34:59.140433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f1939693-2308-481d-9bfa-ffd1a8b34b47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:34:59.231226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1939693-2308-481d-9bfa-ffd1a8b34b47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:34:59.232861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1939693-2308-481d-9bfa-ffd1a8b34b47 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:34:59.251469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 693/1385] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:01.289316 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d592e-1573-48ad-abf4-96bf895bb495 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:01.289995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d592e-1573-48ad-abf4-96bf895bb495 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:01.289995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d592e-1573-48ad-abf4-96bf895bb495 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:01.291240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 693/1386] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:01.296975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d2a07-319b-4caf-947f-1650c867f507 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:01.297433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d2a07-319b-4caf-947f-1650c867f507 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:01.297631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d2a07-319b-4caf-947f-1650c867f507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:01.376985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-949d2a07-319b-4caf-947f-1650c867f507 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:01.378218 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-949d2a07-319b-4caf-947f-1650c867f507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:01.419085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:01.491618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 694/1387] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:01.549536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d592e-1573-48ad-abf4-96bf895bb495 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:01.551589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f58d592e-1573-48ad-abf4-96bf895bb495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:01.668171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:01.742312 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d592e-1573-48ad-abf4-96bf895bb495 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:01.871105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc3e40-c624-4c53-9a9a-5ad1fb231583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:01.871802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc3e40-c624-4c53-9a9a-5ad1fb231583 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:01.872165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc3e40-c624-4c53-9a9a-5ad1fb231583 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:01.872491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc3e40-c624-4c53-9a9a-5ad1fb231583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:01.874415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b8dc3e40-c624-4c53-9a9a-5ad1fb231583 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:01.918086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:01.982393 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc3e40-c624-4c53-9a9a-5ad1fb231583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:01.984303 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc3e40-c624-4c53-9a9a-5ad1fb231583 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:02.006297 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 694/1388] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:04.181441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa457a9-5419-479e-8704-8a016bc2a883 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:04.182013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa457a9-5419-479e-8704-8a016bc2a883 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:04.182013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa457a9-5419-479e-8704-8a016bc2a883 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:04.183345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 695/1389] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:04.188503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a9ab3-c364-47b8-a330-d3b52c5c3137 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:04.188951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a9ab3-c364-47b8-a330-d3b52c5c3137 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:04.189074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a9ab3-c364-47b8-a330-d3b52c5c3137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:04.255766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-398a9ab3-c364-47b8-a330-d3b52c5c3137 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:04.256907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-398a9ab3-c364-47b8-a330-d3b52c5c3137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:04.354406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 695/1390] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:04.406761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa457a9-5419-479e-8704-8a016bc2a883 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:04.408509 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-afa457a9-5419-479e-8704-8a016bc2a883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:04.553243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa457a9-5419-479e-8704-8a016bc2a883 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:04.570059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:04.665414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b0abf-0f68-4246-a379-8efed6c71bd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:04.666026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b0abf-0f68-4246-a379-8efed6c71bd7 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:04.666304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b0abf-0f68-4246-a379-8efed6c71bd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:04.666579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b0abf-0f68-4246-a379-8efed6c71bd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:04.668232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-567b0abf-0f68-4246-a379-8efed6c71bd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:04.757243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b0abf-0f68-4246-a379-8efed6c71bd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:04.758841 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b0abf-0f68-4246-a379-8efed6c71bd7 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:04.777253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 696/1391] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:05.058100 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d1d3-956b-4254-93e9-d477b370608f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:05.058441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d1d3-956b-4254-93e9-d477b370608f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:05.058680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d1d3-956b-4254-93e9-d477b370608f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:05.060078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 696/1392] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:05.065774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b907ef9-db7d-407f-b714-44467b222c8d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:05.066046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b907ef9-db7d-407f-b714-44467b222c8d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:05.066320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b907ef9-db7d-407f-b714-44467b222c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:05.092510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:05.135090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8b907ef9-db7d-407f-b714-44467b222c8d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:05.136241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8b907ef9-db7d-407f-b714-44467b222c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:05.233947 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 697/1393] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:05.291359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea86c68-072a-434b-b9e5-a8640f6a64ae None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:05.291831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea86c68-072a-434b-b9e5-a8640f6a64ae None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:05.291950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea86c68-072a-434b-b9e5-a8640f6a64ae None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:05.293292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 697/1394] 10.222.0.64 () {54 vars in 966 bytes} [Thu Aug 4 07:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:05.298237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92909e-dd4c-4d58-be64-482d649d950e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:05.298532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92909e-dd4c-4d58-be64-482d649d950e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:05.298811 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92909e-dd4c-4d58-be64-482d649d950e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:05.376153 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7f92909e-dd4c-4d58-be64-482d649d950e None None] MFA Rules not processed for user `56dfc4dfda424d40b7c9f17345b8ad45`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:05.377354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7f92909e-dd4c-4d58-be64-482d649d950e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:05.418734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:05.454483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 698/1395] 10.222.0.64 () {58 vars in 1063 bytes} [Thu Aug 4 07:35:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:05.462831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea86c68-072a-434b-b9e5-a8640f6a64ae None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:05.464411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aea86c68-072a-434b-b9e5-a8640f6a64ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:05.478427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:05.570096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea86c68-072a-434b-b9e5-a8640f6a64ae None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:05.646524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680b5c0-1e7c-4d78-aa66-a44a8cc20805 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfc4dfda424d40b7c9f17345b8ad45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:05.647157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680b5c0-1e7c-4d78-aa66-a44a8cc20805 service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:05.647446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680b5c0-1e7c-4d78-aa66-a44a8cc20805 service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:05.647745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680b5c0-1e7c-4d78-aa66-a44a8cc20805 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:05.649452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5680b5c0-1e7c-4d78-aa66-a44a8cc20805 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:05.737038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:05.785388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680b5c0-1e7c-4d78-aa66-a44a8cc20805 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:05.787900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680b5c0-1e7c-4d78-aa66-a44a8cc20805 service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:05.809112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 698/1396] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:06.689706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e1737-773e-4335-840b-1cd3725da2a9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:06.690232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e1737-773e-4335-840b-1cd3725da2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:06.690351 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e1737-773e-4335-840b-1cd3725da2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:06.759164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-857e1737-773e-4335-840b-1cd3725da2a9 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:06.760406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-857e1737-773e-4335-840b-1cd3725da2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:06.847673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 699/1397] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:35:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:06.891346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f600af-6708-4edc-ac15-5bface62bf9e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:06.891858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f600af-6708-4edc-ac15-5bface62bf9e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:06.891966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f600af-6708-4edc-ac15-5bface62bf9e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:06.893373 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 699/1398] 10.222.0.64 () {54 vars in 943 bytes} [Thu Aug 4 07:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:06.900595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548cd7a-5127-41a5-b890-361883a67122 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:06.900961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548cd7a-5127-41a5-b890-361883a67122 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:06.901305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548cd7a-5127-41a5-b890-361883a67122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:06.929059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25ee3-a19f-4d99-b07d-f71a9e7637a8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:06.929403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25ee3-a19f-4d99-b07d-f71a9e7637a8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:06.929688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25ee3-a19f-4d99-b07d-f71a9e7637a8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:06.931021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 700/1399] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:06.937660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027b85b-7946-43f1-9b2d-00dd978987cf None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:06.937660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027b85b-7946-43f1-9b2d-00dd978987cf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:06.937660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027b85b-7946-43f1-9b2d-00dd978987cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:06.975396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:06.982831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e548cd7a-5127-41a5-b890-361883a67122 None None] MFA Rules not processed for user `9503e2888d5e4831afc0d9d9522c3eea`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:06.995106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e548cd7a-5127-41a5-b890-361883a67122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:07.007397 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c027b85b-7946-43f1-9b2d-00dd978987cf None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:07.008637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c027b85b-7946-43f1-9b2d-00dd978987cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:07.104917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 700/1400] 10.222.0.64 () {58 vars in 1040 bytes} [Thu Aug 4 07:35:06 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:07.114191 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548cd7a-5127-41a5-b890-361883a67122 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:07.122740 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 701/1401] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:35:07.124730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548cd7a-5127-41a5-b890-361883a67122 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:07.166679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027b85b-7946-43f1-9b2d-00dd978987cf None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:07.167928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c027b85b-7946-43f1-9b2d-00dd978987cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:07.195571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:07.230561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a80abe-527c-40c1-97d1-076f624aea79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:07.231187 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a80abe-527c-40c1-97d1-076f624aea79 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:07.231496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a80abe-527c-40c1-97d1-076f624aea79 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:07.231798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a80abe-527c-40c1-97d1-076f624aea79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:07.233537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-09a80abe-527c-40c1-97d1-076f624aea79 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:07.236140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:07.286051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027b85b-7946-43f1-9b2d-00dd978987cf None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:07.317314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a80abe-527c-40c1-97d1-076f624aea79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:07.318964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a80abe-527c-40c1-97d1-076f624aea79 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:07.327225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 701/1402] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:35:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 07:35:07.408368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e1e83-8b83-41be-8936-608c5420217b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:07.408904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e1e83-8b83-41be-8936-608c5420217b service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:07.409184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e1e83-8b83-41be-8936-608c5420217b service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:07.409458 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e1e83-8b83-41be-8936-608c5420217b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:07.411059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f82e1e83-8b83-41be-8936-608c5420217b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:07.428181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:07.498875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e1e83-8b83-41be-8936-608c5420217b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:07.500498 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e1e83-8b83-41be-8936-608c5420217b service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:07.518825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 702/1403] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:09.715927 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc2bf-a52d-4fbe-ba81-bba9107c8f99 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:09.716513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc2bf-a52d-4fbe-ba81-bba9107c8f99 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:09.716513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc2bf-a52d-4fbe-ba81-bba9107c8f99 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:09.718398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 702/1404] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:09.723649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a55a8-bf4f-4e09-b784-85c66ba2061a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:09.724610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a55a8-bf4f-4e09-b784-85c66ba2061a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:09.724777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a55a8-bf4f-4e09-b784-85c66ba2061a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:09.800077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2f5a55a8-bf4f-4e09-b784-85c66ba2061a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:09.801313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2f5a55a8-bf4f-4e09-b784-85c66ba2061a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:09.908123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:09.920137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 703/1405] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:09.963066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dc2bf-a52d-4fbe-ba81-bba9107c8f99 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:09.966583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-300dc2bf-a52d-4fbe-ba81-bba9107c8f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:10.128615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:10.131693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dc2bf-a52d-4fbe-ba81-bba9107c8f99 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:10.273059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac422e-7b32-41ae-8481-8fcf9b6178b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:10.273686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac422e-7b32-41ae-8481-8fcf9b6178b7 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:10.273975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac422e-7b32-41ae-8481-8fcf9b6178b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:10.274249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac422e-7b32-41ae-8481-8fcf9b6178b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:10.276027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-20ac422e-7b32-41ae-8481-8fcf9b6178b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:10.398756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac422e-7b32-41ae-8481-8fcf9b6178b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:10.400676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac422e-7b32-41ae-8481-8fcf9b6178b7 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:10.423823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 703/1406] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:13.298085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d10672-64d7-4bc2-b32e-8e302a576336 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:13.298662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d10672-64d7-4bc2-b32e-8e302a576336 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:13.298662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d10672-64d7-4bc2-b32e-8e302a576336 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:13.304211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 704/1407] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:35:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:13.305864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2d6cb-0913-4aef-b838-85db80051f2d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:13.306317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2d6cb-0913-4aef-b838-85db80051f2d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:13.306406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2d6cb-0913-4aef-b838-85db80051f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:13.311907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:13.386008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-13d2d6cb-0913-4aef-b838-85db80051f2d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:13.387322 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-13d2d6cb-0913-4aef-b838-85db80051f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:13.494959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 704/1408] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:13.541832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d10672-64d7-4bc2-b32e-8e302a576336 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:13.543453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e6d10672-64d7-4bc2-b32e-8e302a576336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:13.710983 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d10672-64d7-4bc2-b32e-8e302a576336 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:13.785650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:13.843563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20059a47-d163-4463-aa19-5d426e03f1ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:13.844223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20059a47-d163-4463-aa19-5d426e03f1ad service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:13.844509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20059a47-d163-4463-aa19-5d426e03f1ad service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:13.844786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20059a47-d163-4463-aa19-5d426e03f1ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:13.846468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-20059a47-d163-4463-aa19-5d426e03f1ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:13.943250 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20059a47-d163-4463-aa19-5d426e03f1ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:13.945279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20059a47-d163-4463-aa19-5d426e03f1ad service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:13.968823 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 705/1409] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:23.493166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=64 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:23.966156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=64 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:30.782070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daa604-e000-4468-9db1-3067d30b5258 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:30.782667 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daa604-e000-4468-9db1-3067d30b5258 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:30.782667 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daa604-e000-4468-9db1-3067d30b5258 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:30.783950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 705/1410] 10.222.0.64 () {54 vars in 918 bytes} [Thu Aug 4 07:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:30.789572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff801e-449c-402f-8125-226c811d6ada None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:30.790042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff801e-449c-402f-8125-226c811d6ada None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:30.790156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff801e-449c-402f-8125-226c811d6ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:30.990794 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bfff801e-449c-402f-8125-226c811d6ada None None] MFA Rules not processed for user `affaeff3cd294584b7238909e0457f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:30.992091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bfff801e-449c-402f-8125-226c811d6ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:31.100152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 706/1411] 10.222.0.64 () {58 vars in 1015 bytes} [Thu Aug 4 07:35:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:35:31.114697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb9b12-5942-4a3b-bd29-b61afa1e5471 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:31.114974 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb9b12-5942-4a3b-bd29-b61afa1e5471 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:31.115289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb9b12-5942-4a3b-bd29-b61afa1e5471 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:31.116668 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 706/1412] 10.222.0.64 () {54 vars in 918 bytes} [Thu Aug 4 07:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:31.121375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1731538-88c0-42c0-a508-495d8fe451ee None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:31.121664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1731538-88c0-42c0-a508-495d8fe451ee None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:31.121944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1731538-88c0-42c0-a508-495d8fe451ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:31.193730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:31.196317 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c1731538-88c0-42c0-a508-495d8fe451ee None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:31.197537 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c1731538-88c0-42c0-a508-495d8fe451ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:31.311706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 707/1413] 10.222.0.64 () {58 vars in 1015 bytes} [Thu Aug 4 07:35:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:33.814786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef5c9c-561a-4b9d-8d41-36de9e8bbd25 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:33.815338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef5c9c-561a-4b9d-8d41-36de9e8bbd25 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:33.815452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef5c9c-561a-4b9d-8d41-36de9e8bbd25 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:33.816836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:33.822293 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd2b9-9436-4278-b49d-20477dd6966d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:33.822678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd2b9-9436-4278-b49d-20477dd6966d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:33.822873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd2b9-9436-4278-b49d-20477dd6966d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:33.889644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9d2dd2b9-9436-4278-b49d-20477dd6966d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:33.890854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9d2dd2b9-9436-4278-b49d-20477dd6966d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:33.945418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:33.987317 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:34.030045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef5c9c-561a-4b9d-8d41-36de9e8bbd25 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:34.031559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-daef5c9c-561a-4b9d-8d41-36de9e8bbd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:34.174677 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef5c9c-561a-4b9d-8d41-36de9e8bbd25 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:34.246040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:34.285075 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab4f92-c1cb-4f75-907f-2e39a2c203f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:34.285712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab4f92-c1cb-4f75-907f-2e39a2c203f1 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:34.285996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab4f92-c1cb-4f75-907f-2e39a2c203f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:34.286272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab4f92-c1cb-4f75-907f-2e39a2c203f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:34.287885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0cab4f92-c1cb-4f75-907f-2e39a2c203f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:34.374433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab4f92-c1cb-4f75-907f-2e39a2c203f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:34.376059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab4f92-c1cb-4f75-907f-2e39a2c203f1 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:34.394273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 708/1416] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:35.295462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d3935-df0c-4e53-8442-083e2bc0fb8d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:35.295462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d3935-df0c-4e53-8442-083e2bc0fb8d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:35.295462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d3935-df0c-4e53-8442-083e2bc0fb8d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:35.295462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:35.298558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6401-1e91-471d-b989-25340d2573b9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:35.298942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6401-1e91-471d-b989-25340d2573b9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:35.299100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6401-1e91-471d-b989-25340d2573b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:35.374299 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9e9e6401-1e91-471d-b989-25340d2573b9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:35.375485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e9e6401-1e91-471d-b989-25340d2573b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:35.401237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:35.470340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:35.620521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d3935-df0c-4e53-8442-083e2bc0fb8d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:35.621961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-181d3935-df0c-4e53-8442-083e2bc0fb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:35.764151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d3935-df0c-4e53-8442-083e2bc0fb8d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:35.803053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:35.874400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de965d-5510-49b1-8f8f-87fae011aceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:35.874939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de965d-5510-49b1-8f8f-87fae011aceb service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:35.875216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de965d-5510-49b1-8f8f-87fae011aceb service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:35.875488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de965d-5510-49b1-8f8f-87fae011aceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:35.877126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-40de965d-5510-49b1-8f8f-87fae011aceb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:35.965398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de965d-5510-49b1-8f8f-87fae011aceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:35.966996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de965d-5510-49b1-8f8f-87fae011aceb service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:35.984987 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 710/1419] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:36.872089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8765f-c819-4655-8663-bb6e5e64d303 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:36.872695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8765f-c819-4655-8663-bb6e5e64d303 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:36.872695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8765f-c819-4655-8663-bb6e5e64d303 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:36.874001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:36.879229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13603fe5-d67a-42da-9797-13634e12e03e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:36.879659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13603fe5-d67a-42da-9797-13634e12e03e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:36.879795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13603fe5-d67a-42da-9797-13634e12e03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:36.934882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:36.957594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-13603fe5-d67a-42da-9797-13634e12e03e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:36.958820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-13603fe5-d67a-42da-9797-13634e12e03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:37.054654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:37.210443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8765f-c819-4655-8663-bb6e5e64d303 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:37.212008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-baa8765f-c819-4655-8663-bb6e5e64d303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:37.355519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8765f-c819-4655-8663-bb6e5e64d303 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:37.367528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:37.466590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c61bb-2112-4dd7-a21c-74f3217c51b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:37.467122 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c61bb-2112-4dd7-a21c-74f3217c51b0 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:37.467405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c61bb-2112-4dd7-a21c-74f3217c51b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:37.467945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c61bb-2112-4dd7-a21c-74f3217c51b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:37.469623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9d1c61bb-2112-4dd7-a21c-74f3217c51b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:37.558631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c61bb-2112-4dd7-a21c-74f3217c51b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:37.560248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c61bb-2112-4dd7-a21c-74f3217c51b0 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:37.578709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 711/1422] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:38.445153 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c487b9e-aae4-476d-a63f-3de59ed6dba9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:38.445725 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c487b9e-aae4-476d-a63f-3de59ed6dba9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:38.445725 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c487b9e-aae4-476d-a63f-3de59ed6dba9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:38.447093 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:38.452162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf2ed0-7ca2-4d27-b861-b2b6210a26b7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:38.452605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf2ed0-7ca2-4d27-b861-b2b6210a26b7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:38.452714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf2ed0-7ca2-4d27-b861-b2b6210a26b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:38.472221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:38.528696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1adf2ed0-7ca2-4d27-b861-b2b6210a26b7 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:38.529896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1adf2ed0-7ca2-4d27-b861-b2b6210a26b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:38.625252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:38.776140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c487b9e-aae4-476d-a63f-3de59ed6dba9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:38.777616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4c487b9e-aae4-476d-a63f-3de59ed6dba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:38.887263 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:38.921604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c487b9e-aae4-476d-a63f-3de59ed6dba9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:39.033398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd8bf0-55d6-42a8-96cb-f9f8c79b5ebc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:39.033946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd8bf0-55d6-42a8-96cb-f9f8c79b5ebc service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:39.034229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd8bf0-55d6-42a8-96cb-f9f8c79b5ebc service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:39.034504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd8bf0-55d6-42a8-96cb-f9f8c79b5ebc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:39.036357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6bdd8bf0-55d6-42a8-96cb-f9f8c79b5ebc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:39.112108 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:39.125105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd8bf0-55d6-42a8-96cb-f9f8c79b5ebc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:39.126687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd8bf0-55d6-42a8-96cb-f9f8c79b5ebc service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:39.145423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 713/1425] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:40.005083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d58ec8-0232-409c-857a-3c97ca7ef977 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:40.005604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d58ec8-0232-409c-857a-3c97ca7ef977 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:40.005727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d58ec8-0232-409c-857a-3c97ca7ef977 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:40.006981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:35:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:40.011935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e01b5-1df6-4101-a2f5-bf12d0a5e664 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:40.012378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e01b5-1df6-4101-a2f5-bf12d0a5e664 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:40.012492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e01b5-1df6-4101-a2f5-bf12d0a5e664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:40.088728 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3f3e01b5-1df6-4101-a2f5-bf12d0a5e664 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:40.089941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3f3e01b5-1df6-4101-a2f5-bf12d0a5e664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:40.184459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:40.337574 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d58ec8-0232-409c-857a-3c97ca7ef977 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:40.339026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-77d58ec8-0232-409c-857a-3c97ca7ef977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:40.421489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:40.482938 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d58ec8-0232-409c-857a-3c97ca7ef977 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:40.592924 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85096a31-f801-4d4a-b9b8-cdfbf7950b38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:40.593459 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85096a31-f801-4d4a-b9b8-cdfbf7950b38 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:40.593743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85096a31-f801-4d4a-b9b8-cdfbf7950b38 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:40.594017 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85096a31-f801-4d4a-b9b8-cdfbf7950b38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:40.595631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-85096a31-f801-4d4a-b9b8-cdfbf7950b38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:40.642438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:40.682805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85096a31-f801-4d4a-b9b8-cdfbf7950b38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:40.684441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85096a31-f801-4d4a-b9b8-cdfbf7950b38 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:40.702643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 714/1428] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:41.557143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e20b81-e7a9-4f56-8bd4-c0aadeeed9d3 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:41.557691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e20b81-e7a9-4f56-8bd4-c0aadeeed9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:41.557691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e20b81-e7a9-4f56-8bd4-c0aadeeed9d3 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:41.559043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:41.563960 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f25b2-7bb7-4d3b-bd3a-b39479a008b4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:41.564417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f25b2-7bb7-4d3b-bd3a-b39479a008b4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:41.564541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f25b2-7bb7-4d3b-bd3a-b39479a008b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:41.639356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-186f25b2-7bb7-4d3b-bd3a-b39479a008b4 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:41.640576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-186f25b2-7bb7-4d3b-bd3a-b39479a008b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:41.736366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:41.892385 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e20b81-e7a9-4f56-8bd4-c0aadeeed9d3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:41.894076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-34e20b81-e7a9-4f56-8bd4-c0aadeeed9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:41.948009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:42.043609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e20b81-e7a9-4f56-8bd4-c0aadeeed9d3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:42.154128 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e4049-4108-4681-ad58-d9502967036a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:42.154664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e4049-4108-4681-ad58-d9502967036a service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:42.154941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e4049-4108-4681-ad58-d9502967036a service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:42.155212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e4049-4108-4681-ad58-d9502967036a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:42.156852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d65e4049-4108-4681-ad58-d9502967036a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:42.163680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:42.243249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e4049-4108-4681-ad58-d9502967036a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:42.244851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e4049-4108-4681-ad58-d9502967036a service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:42.262628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 716/1431] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:43.133283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f9c57-77ab-454a-aecb-cf96abd199a8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:43.133855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f9c57-77ab-454a-aecb-cf96abd199a8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:43.133855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f9c57-77ab-454a-aecb-cf96abd199a8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:43.135165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:43.140115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318fbc2-d554-4dab-b56e-fd27e51812c2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:43.140535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318fbc2-d554-4dab-b56e-fd27e51812c2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:43.140676 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318fbc2-d554-4dab-b56e-fd27e51812c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:43.216333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d318fbc2-d554-4dab-b56e-fd27e51812c2 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:43.217739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d318fbc2-d554-4dab-b56e-fd27e51812c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:43.283158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:43.311761 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:43.464704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f9c57-77ab-454a-aecb-cf96abd199a8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:43.466247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-258f9c57-77ab-454a-aecb-cf96abd199a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:43.483895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:43.608849 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f9c57-77ab-454a-aecb-cf96abd199a8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:43.693813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:43.719257 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb1728-1c85-4a56-8dc0-417f4e671efd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:43.719866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb1728-1c85-4a56-8dc0-417f4e671efd service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:43.720537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb1728-1c85-4a56-8dc0-417f4e671efd service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:43.720537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb1728-1c85-4a56-8dc0-417f4e671efd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:43.722148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fcfb1728-1c85-4a56-8dc0-417f4e671efd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:43.809655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb1728-1c85-4a56-8dc0-417f4e671efd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:43.811173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb1728-1c85-4a56-8dc0-417f4e671efd service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:43.829282 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 717/1434] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:44.741261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7a13d-3819-442c-8044-689a6c80db4d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:44.741833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7a13d-3819-442c-8044-689a6c80db4d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:44.741833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7a13d-3819-442c-8044-689a6c80db4d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:44.743155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:44.748011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe809f40-2595-443a-997e-ea69351c52d8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:44.748417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe809f40-2595-443a-997e-ea69351c52d8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:44.748588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe809f40-2595-443a-997e-ea69351c52d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:44.823821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fe809f40-2595-443a-997e-ea69351c52d8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:44.825047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fe809f40-2595-443a-997e-ea69351c52d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:44.866241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:44.920854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:45.074606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7a13d-3819-442c-8044-689a6c80db4d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:45.076241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-81c7a13d-3819-442c-8044-689a6c80db4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:45.220708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7a13d-3819-442c-8044-689a6c80db4d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:45.274116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:45.331829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9c903-ecec-485a-8529-e6beff423885 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:45.332470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9c903-ecec-485a-8529-e6beff423885 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:45.332746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9c903-ecec-485a-8529-e6beff423885 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:45.333012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9c903-ecec-485a-8529-e6beff423885 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:45.334642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e6c9c903-ecec-485a-8529-e6beff423885 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:45.420163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9c903-ecec-485a-8529-e6beff423885 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:45.421765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9c903-ecec-485a-8529-e6beff423885 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:45.439556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 719/1437] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:46.305368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daff20d-56b0-4771-acc9-d86bbf9da36a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:46.305904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daff20d-56b0-4771-acc9-d86bbf9da36a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:46.305904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daff20d-56b0-4771-acc9-d86bbf9da36a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:46.307208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:46.312124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e269f-4e29-43fe-a46b-66aa6dfc9205 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:46.312570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e269f-4e29-43fe-a46b-66aa6dfc9205 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:46.312690 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e269f-4e29-43fe-a46b-66aa6dfc9205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:46.389003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fb4e269f-4e29-43fe-a46b-66aa6dfc9205 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:46.390211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fb4e269f-4e29-43fe-a46b-66aa6dfc9205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:46.392273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:46.485354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:46.639938 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daff20d-56b0-4771-acc9-d86bbf9da36a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:46.641523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3daff20d-56b0-4771-acc9-d86bbf9da36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:46.789176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daff20d-56b0-4771-acc9-d86bbf9da36a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:46.814723 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:46.901498 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b55f36-2ad8-4f60-b8bc-78e4961ff8ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:46.902020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b55f36-2ad8-4f60-b8bc-78e4961ff8ea service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:46.902303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b55f36-2ad8-4f60-b8bc-78e4961ff8ea service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:46.902575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b55f36-2ad8-4f60-b8bc-78e4961ff8ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:46.904263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f7b55f36-2ad8-4f60-b8bc-78e4961ff8ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:46.994760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b55f36-2ad8-4f60-b8bc-78e4961ff8ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:46.996460 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b55f36-2ad8-4f60-b8bc-78e4961ff8ea service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:47.017015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 720/1440] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:48.935527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5bbc6-c7f3-48fd-af05-89c9b1f05596 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:48.936151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5bbc6-c7f3-48fd-af05-89c9b1f05596 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:48.936151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5bbc6-c7f3-48fd-af05-89c9b1f05596 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:48.937432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:48.943203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3d798-5c8c-4e61-a020-89469bbf10e6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:48.943655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3d798-5c8c-4e61-a020-89469bbf10e6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:48.943748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3d798-5c8c-4e61-a020-89469bbf10e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:48.987168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:49.017730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-05e3d798-5c8c-4e61-a020-89469bbf10e6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:49.019443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-05e3d798-5c8c-4e61-a020-89469bbf10e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:49.129112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:49.171132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5bbc6-c7f3-48fd-af05-89c9b1f05596 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:49.172864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4be5bbc6-c7f3-48fd-af05-89c9b1f05596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:49.287461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5bbc6-c7f3-48fd-af05-89c9b1f05596 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:49.327368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:49.372546 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d482ac-683e-4271-ac88-4b5cfebeeaa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:49.373295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d482ac-683e-4271-ac88-4b5cfebeeaa1 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:49.373583 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d482ac-683e-4271-ac88-4b5cfebeeaa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:49.373862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d482ac-683e-4271-ac88-4b5cfebeeaa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:49.375508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-70d482ac-683e-4271-ac88-4b5cfebeeaa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:49.497001 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d482ac-683e-4271-ac88-4b5cfebeeaa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:49.499009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d482ac-683e-4271-ac88-4b5cfebeeaa1 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:49.521747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 722/1443] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:51.693055 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704edc9-8d32-494e-be6b-7d5317baa3c4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:51.693575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704edc9-8d32-494e-be6b-7d5317baa3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:51.693673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704edc9-8d32-494e-be6b-7d5317baa3c4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:51.694937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:51.700125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116a754-711c-429f-8c5c-df3e53861535 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:51.700559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116a754-711c-429f-8c5c-df3e53861535 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:51.700716 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116a754-711c-429f-8c5c-df3e53861535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:51.766132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0116a754-711c-429f-8c5c-df3e53861535 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:51.767337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0116a754-711c-429f-8c5c-df3e53861535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:51.780517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:51.862279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:51.900917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704edc9-8d32-494e-be6b-7d5317baa3c4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:51.902377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5704edc9-8d32-494e-be6b-7d5317baa3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:52.005635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704edc9-8d32-494e-be6b-7d5317baa3c4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:52.017556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:52.078882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b97b2f-bd3a-443a-8842-4be95d599b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:52.079497 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b97b2f-bd3a-443a-8842-4be95d599b2e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:52.079778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b97b2f-bd3a-443a-8842-4be95d599b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:52.080089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b97b2f-bd3a-443a-8842-4be95d599b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:52.081719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-03b97b2f-bd3a-443a-8842-4be95d599b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:52.189200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b97b2f-bd3a-443a-8842-4be95d599b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:52.190647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b97b2f-bd3a-443a-8842-4be95d599b2e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:52.208846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 723/1446] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:54.568581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bf033-34b2-4711-8daa-ec0210f83071 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:54.569175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bf033-34b2-4711-8daa-ec0210f83071 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:54.569175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bf033-34b2-4711-8daa-ec0210f83071 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:54.570486 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:54.575705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760a11-8a57-4561-9c9d-d551aecdf818 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:54.576128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760a11-8a57-4561-9c9d-d551aecdf818 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:54.576325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760a11-8a57-4561-9c9d-d551aecdf818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:54.624512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:54.642684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e4760a11-8a57-4561-9c9d-d551aecdf818 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:54.644564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e4760a11-8a57-4561-9c9d-d551aecdf818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:54.738103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:54.778210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bf033-34b2-4711-8daa-ec0210f83071 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:54.779776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-736bf033-34b2-4711-8daa-ec0210f83071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:54.882504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bf033-34b2-4711-8daa-ec0210f83071 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:54.922413 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:54.957093 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50517007-e1bc-4634-b277-60491765708e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:54.957770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50517007-e1bc-4634-b277-60491765708e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:54.958077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50517007-e1bc-4634-b277-60491765708e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:54.958350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50517007-e1bc-4634-b277-60491765708e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:54.960026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-50517007-e1bc-4634-b277-60491765708e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:55.068485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50517007-e1bc-4634-b277-60491765708e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:55.069963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50517007-e1bc-4634-b277-60491765708e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:55.089083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 725/1449] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:57.237430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa74bf-ce43-431f-b818-60a24180f1a0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:57.237958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa74bf-ce43-431f-b818-60a24180f1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:57.237958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa74bf-ce43-431f-b818-60a24180f1a0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:57.239240 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:57.244600 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05897d22-43f3-4d63-b2fd-b6d16c55fe00 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:57.245032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05897d22-43f3-4d63-b2fd-b6d16c55fe00 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:57.245129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05897d22-43f3-4d63-b2fd-b6d16c55fe00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:57.310707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-05897d22-43f3-4d63-b2fd-b6d16c55fe00 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:57.311917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-05897d22-43f3-4d63-b2fd-b6d16c55fe00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:57.332419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:57.408817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:35:57.449694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa74bf-ce43-431f-b818-60a24180f1a0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:35:57.451214 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-55aa74bf-ce43-431f-b818-60a24180f1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:57.553325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa74bf-ce43-431f-b818-60a24180f1a0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:35:57.573587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:57.625800 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31fe1c-7d65-411f-aa00-7a5428d6537e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:35:57.626413 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31fe1c-7d65-411f-aa00-7a5428d6537e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:57.626693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31fe1c-7d65-411f-aa00-7a5428d6537e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:57.626965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31fe1c-7d65-411f-aa00-7a5428d6537e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:57.629650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ca31fe1c-7d65-411f-aa00-7a5428d6537e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:35:57.735820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31fe1c-7d65-411f-aa00-7a5428d6537e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:35:57.737380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31fe1c-7d65-411f-aa00-7a5428d6537e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:35:57.756152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 726/1452] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:35:59.905791 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220a57c-3f5e-4d5d-9a24-ac58e1141407 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:59.906327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220a57c-3f5e-4d5d-9a24-ac58e1141407 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:59.906327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220a57c-3f5e-4d5d-9a24-ac58e1141407 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:59.907600 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:35:59.913431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae93e5-a913-457c-b68b-057d088f66dc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:35:59.913846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae93e5-a913-457c-b68b-057d088f66dc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:35:59.913971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae93e5-a913-457c-b68b-057d088f66dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:35:59.978183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:35:59.981087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-77ae93e5-a913-457c-b68b-057d088f66dc None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:35:59.982301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-77ae93e5-a913-457c-b68b-057d088f66dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:00.079070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:00.117695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220a57c-3f5e-4d5d-9a24-ac58e1141407 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:00.119228 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1220a57c-3f5e-4d5d-9a24-ac58e1141407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:00.224427 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220a57c-3f5e-4d5d-9a24-ac58e1141407 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:00.272848 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:00.297023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d71b68-e007-4066-98af-56c8b11af160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:00.297635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d71b68-e007-4066-98af-56c8b11af160 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:00.297916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d71b68-e007-4066-98af-56c8b11af160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:00.298198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d71b68-e007-4066-98af-56c8b11af160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:00.301466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-25d71b68-e007-4066-98af-56c8b11af160 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:00.408340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d71b68-e007-4066-98af-56c8b11af160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:00.409793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d71b68-e007-4066-98af-56c8b11af160 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:00.428219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 728/1455] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:02.586034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0f692-0ff2-4c65-a751-757485bcc785 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:02.586588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0f692-0ff2-4c65-a751-757485bcc785 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:02.586588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0f692-0ff2-4c65-a751-757485bcc785 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:02.587936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:02.593833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2182d9-5c67-4821-9085-dd24ae686601 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:02.594257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2182d9-5c67-4821-9085-dd24ae686601 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:02.594354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2182d9-5c67-4821-9085-dd24ae686601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:02.659832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dd2182d9-5c67-4821-9085-dd24ae686601 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:02.661086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dd2182d9-5c67-4821-9085-dd24ae686601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:02.689439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:02.757928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:36:02.795952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0f692-0ff2-4c65-a751-757485bcc785 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:02.797544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-40a0f692-0ff2-4c65-a751-757485bcc785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:02.900085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0f692-0ff2-4c65-a751-757485bcc785 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:02.928587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:02.973668 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61de412-3a0e-4a95-a03a-14599fa3fea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:02.974282 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61de412-3a0e-4a95-a03a-14599fa3fea0 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:02.974563 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61de412-3a0e-4a95-a03a-14599fa3fea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:02.974839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61de412-3a0e-4a95-a03a-14599fa3fea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:02.976484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b61de412-3a0e-4a95-a03a-14599fa3fea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:03.083936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61de412-3a0e-4a95-a03a-14599fa3fea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:03.085414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61de412-3a0e-4a95-a03a-14599fa3fea0 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:03.104733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 729/1458] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:05.152979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02330d-c36f-4189-9403-1a4b8b5932b2 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:05.153586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02330d-c36f-4189-9403-1a4b8b5932b2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:05.153586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02330d-c36f-4189-9403-1a4b8b5932b2 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:05.154931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:05.160844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad0253-82dd-4689-9dcd-e53f79de4797 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:05.161276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad0253-82dd-4689-9dcd-e53f79de4797 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:05.161387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad0253-82dd-4689-9dcd-e53f79de4797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:05.225703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f4ad0253-82dd-4689-9dcd-e53f79de4797 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:05.226804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f4ad0253-82dd-4689-9dcd-e53f79de4797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:05.234845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:05.322684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:05.361559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02330d-c36f-4189-9403-1a4b8b5932b2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:05.363832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ae02330d-c36f-4189-9403-1a4b8b5932b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:05.464061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02330d-c36f-4189-9403-1a4b8b5932b2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:05.517613 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:05.536622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609a56-642a-4cd9-8fec-a297f9cca3f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:05.537233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609a56-642a-4cd9-8fec-a297f9cca3f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:05.537516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609a56-642a-4cd9-8fec-a297f9cca3f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:05.537882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609a56-642a-4cd9-8fec-a297f9cca3f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:05.539491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6f609a56-642a-4cd9-8fec-a297f9cca3f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:05.644241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f609a56-642a-4cd9-8fec-a297f9cca3f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:05.645673 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f609a56-642a-4cd9-8fec-a297f9cca3f8 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:05.664872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 731/1461] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:07.712860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc901d3-1c99-4139-ae6e-7af9c5ea1e7c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:07.713448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc901d3-1c99-4139-ae6e-7af9c5ea1e7c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:07.713448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc901d3-1c99-4139-ae6e-7af9c5ea1e7c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:07.714790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:07.720388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859b905-b20b-4416-83a0-7a3c0f271e78 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:07.720816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859b905-b20b-4416-83a0-7a3c0f271e78 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:07.720946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859b905-b20b-4416-83a0-7a3c0f271e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:07.785483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1859b905-b20b-4416-83a0-7a3c0f271e78 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:07.786678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1859b905-b20b-4416-83a0-7a3c0f271e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:07.824425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:07.881252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:07.919265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc901d3-1c99-4139-ae6e-7af9c5ea1e7c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:07.920853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0fc901d3-1c99-4139-ae6e-7af9c5ea1e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:08.024809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc901d3-1c99-4139-ae6e-7af9c5ea1e7c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:08.061765 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:08.099542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfd3f5-cc29-495f-ac33-5cfa42690025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:08.100189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfd3f5-cc29-495f-ac33-5cfa42690025 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:08.100466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfd3f5-cc29-495f-ac33-5cfa42690025 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:08.100732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfd3f5-cc29-495f-ac33-5cfa42690025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:08.102328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7dbfd3f5-cc29-495f-ac33-5cfa42690025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:08.210449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfd3f5-cc29-495f-ac33-5cfa42690025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:08.211880 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfd3f5-cc29-495f-ac33-5cfa42690025 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:08.230797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 732/1464] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:10.257900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133599e-b9fb-4ef9-9bc4-f79998f09d56 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:10.258432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133599e-b9fb-4ef9-9bc4-f79998f09d56 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:10.258432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133599e-b9fb-4ef9-9bc4-f79998f09d56 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:10.259781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:10.265083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f546-16a1-4f12-8363-dbb68ce79608 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:10.265534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f546-16a1-4f12-8363-dbb68ce79608 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:10.265657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f546-16a1-4f12-8363-dbb68ce79608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:10.331704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-13d9f546-16a1-4f12-8363-dbb68ce79608 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:10.332943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-13d9f546-16a1-4f12-8363-dbb68ce79608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:10.350757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:10.428429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:10.464794 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133599e-b9fb-4ef9-9bc4-f79998f09d56 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:10.466335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2133599e-b9fb-4ef9-9bc4-f79998f09d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:10.569672 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133599e-b9fb-4ef9-9bc4-f79998f09d56 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:10.636515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:10.643171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969599a3-3a9e-4b01-bf18-973dde245962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:10.644480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969599a3-3a9e-4b01-bf18-973dde245962 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:10.644651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969599a3-3a9e-4b01-bf18-973dde245962 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:10.644927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969599a3-3a9e-4b01-bf18-973dde245962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:10.646549 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-969599a3-3a9e-4b01-bf18-973dde245962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:10.754995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969599a3-3a9e-4b01-bf18-973dde245962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:10.756500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969599a3-3a9e-4b01-bf18-973dde245962 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:10.775544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 734/1467] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:12.805292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c1e59-eda2-48ca-a9eb-db745b758e6d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:12.805865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c1e59-eda2-48ca-a9eb-db745b758e6d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:12.805865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c1e59-eda2-48ca-a9eb-db745b758e6d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:12.807243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:12.812580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad62c5c-572b-4092-8f59-9625b34a3d20 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:12.812959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad62c5c-572b-4092-8f59-9625b34a3d20 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:12.813160 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad62c5c-572b-4092-8f59-9625b34a3d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:12.879231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3ad62c5c-572b-4092-8f59-9625b34a3d20 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:12.880465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3ad62c5c-572b-4092-8f59-9625b34a3d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:12.927232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:12.980416 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:13.017600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c1e59-eda2-48ca-a9eb-db745b758e6d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:13.019902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e73c1e59-eda2-48ca-a9eb-db745b758e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:13.124793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c1e59-eda2-48ca-a9eb-db745b758e6d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:13.173235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:13.202168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b699c9a-fc46-467b-bf78-2b63113f391a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:13.202777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b699c9a-fc46-467b-bf78-2b63113f391a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:13.203052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b699c9a-fc46-467b-bf78-2b63113f391a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:13.203325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b699c9a-fc46-467b-bf78-2b63113f391a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:13.204950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5b699c9a-fc46-467b-bf78-2b63113f391a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:13.315996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b699c9a-fc46-467b-bf78-2b63113f391a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:13.317476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b699c9a-fc46-467b-bf78-2b63113f391a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:13.335086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 735/1470] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:22.978589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=22 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:23.332912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=66 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:32.908706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad908ace-a3ce-4b12-9627-d04cebb33cd9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:32.909308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad908ace-a3ce-4b12-9627-d04cebb33cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:32.909308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad908ace-a3ce-4b12-9627-d04cebb33cd9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:32.910610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 736/1471] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:36:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:32.915743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1f2e7-09f9-465d-adeb-a2b423c86ec3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:32.916218 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1f2e7-09f9-465d-adeb-a2b423c86ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:32.916333 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1f2e7-09f9-465d-adeb-a2b423c86ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:32.983520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-82a1f2e7-09f9-465d-adeb-a2b423c86ec3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:32.984771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-82a1f2e7-09f9-465d-adeb-a2b423c86ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:33.080048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 736/1472] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:33.164755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e244079-b225-48e8-b1ec-3a917ffccbf9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:33.165046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e244079-b225-48e8-b1ec-3a917ffccbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:33.165322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e244079-b225-48e8-b1ec-3a917ffccbf9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:33.166651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 737/1473] 10.222.0.64 () {54 vars in 966 bytes} [Thu Aug 4 07:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:33.171402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b7b1-62a7-4691-95fd-c8ad8dff0fcc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:33.171688 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b7b1-62a7-4691-95fd-c8ad8dff0fcc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:33.171965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b7b1-62a7-4691-95fd-c8ad8dff0fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:33.247191 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ba57b7b1-62a7-4691-95fd-c8ad8dff0fcc None None] MFA Rules not processed for user `56dfc4dfda424d40b7c9f17345b8ad45`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:33.248430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ba57b7b1-62a7-4691-95fd-c8ad8dff0fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:33.269582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:33.325164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 737/1474] 10.222.0.64 () {58 vars in 1063 bytes} [Thu Aug 4 07:36:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:33.332889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e244079-b225-48e8-b1ec-3a917ffccbf9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:33.334441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0e244079-b225-48e8-b1ec-3a917ffccbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:33.436688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e244079-b225-48e8-b1ec-3a917ffccbf9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:33.510645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258775e-69f1-4e3b-9be3-156d128ba81d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfc4dfda424d40b7c9f17345b8ad45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:33.511260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258775e-69f1-4e3b-9be3-156d128ba81d service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:33.511544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258775e-69f1-4e3b-9be3-156d128ba81d service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:33.511822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258775e-69f1-4e3b-9be3-156d128ba81d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:33.513476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7258775e-69f1-4e3b-9be3-156d128ba81d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:33.566689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:33.620679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258775e-69f1-4e3b-9be3-156d128ba81d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:33.622122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258775e-69f1-4e3b-9be3-156d128ba81d service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:33.641236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 738/1475] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:35.760246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a711-e83e-420e-8fd0-afbc8a91b167 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:35.760825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a711-e83e-420e-8fd0-afbc8a91b167 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:35.760825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a711-e83e-420e-8fd0-afbc8a91b167 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:35.762148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 738/1476] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:35.767946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34ebe4-75bb-4f0d-9172-1cf47863f86f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:35.768401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34ebe4-75bb-4f0d-9172-1cf47863f86f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:35.768541 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34ebe4-75bb-4f0d-9172-1cf47863f86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:35.833430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4f34ebe4-75bb-4f0d-9172-1cf47863f86f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:35.834640 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4f34ebe4-75bb-4f0d-9172-1cf47863f86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:35.930226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 739/1477] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:35.953779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9a711-e83e-420e-8fd0-afbc8a91b167 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:35.956115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-13c9a711-e83e-420e-8fd0-afbc8a91b167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:36.078942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9a711-e83e-420e-8fd0-afbc8a91b167 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:36.127422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:36.171615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86ccb5-24e2-43fd-9e9b-3b22d9a7915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:36.172250 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86ccb5-24e2-43fd-9e9b-3b22d9a7915d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:36.172531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86ccb5-24e2-43fd-9e9b-3b22d9a7915d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:36.172802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86ccb5-24e2-43fd-9e9b-3b22d9a7915d admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:36.173685 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86ccb5-24e2-43fd-9e9b-3b22d9a7915d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:36.175435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-da86ccb5-24e2-43fd-9e9b-3b22d9a7915d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:36:36.175435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Aug 04 07:36:36.175435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: Aug 04 07:36:36.176758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86ccb5-24e2-43fd-9e9b-3b22d9a7915d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:36.195974 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 739/1478] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Aug 4 07:36:35 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:36:38.230340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df26e1-d29f-4a09-ab94-ec5714ac2475 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:38.230954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df26e1-d29f-4a09-ab94-ec5714ac2475 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:38.230954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df26e1-d29f-4a09-ab94-ec5714ac2475 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:38.232222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 740/1479] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:38.237630 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a1444-f227-4a7d-ab7f-395f5e1c6db5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:38.238061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a1444-f227-4a7d-ab7f-395f5e1c6db5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:38.238188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a1444-f227-4a7d-ab7f-395f5e1c6db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:38.302193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f36a1444-f227-4a7d-ab7f-395f5e1c6db5 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:38.303382 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f36a1444-f227-4a7d-ab7f-395f5e1c6db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:38.400350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 740/1480] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:38.443192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df26e1-d29f-4a09-ab94-ec5714ac2475 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:38.444742 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-56df26e1-d29f-4a09-ab94-ec5714ac2475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:38.454331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:38.590647 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df26e1-d29f-4a09-ab94-ec5714ac2475 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:38.670668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:38.704206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c6bce-7443-450f-815b-3cc9ba62075f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:38.704834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c6bce-7443-450f-815b-3cc9ba62075f service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:38.705116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c6bce-7443-450f-815b-3cc9ba62075f service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:38.705391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c6bce-7443-450f-815b-3cc9ba62075f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:38.707018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0a0c6bce-7443-450f-815b-3cc9ba62075f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:38.794745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c6bce-7443-450f-815b-3cc9ba62075f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:38.796379 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c6bce-7443-450f-815b-3cc9ba62075f service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:38.815195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 741/1481] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:41.000989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59a139-8b2d-472b-9d4c-88cfbf9f99a4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:41.002129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59a139-8b2d-472b-9d4c-88cfbf9f99a4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:41.002129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59a139-8b2d-472b-9d4c-88cfbf9f99a4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:41.003600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 741/1482] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:41.009123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fd380-8df1-4142-a33c-ca4a44628606 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:41.009556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fd380-8df1-4142-a33c-ca4a44628606 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:41.009689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fd380-8df1-4142-a33c-ca4a44628606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:41.077587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-df7fd380-8df1-4142-a33c-ca4a44628606 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:41.078901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-df7fd380-8df1-4142-a33c-ca4a44628606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:41.127975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:41.178364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 742/1483] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:41.215537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59a139-8b2d-472b-9d4c-88cfbf9f99a4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:41.217353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-be59a139-8b2d-472b-9d4c-88cfbf9f99a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:41.242063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:41.322126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59a139-8b2d-472b-9d4c-88cfbf9f99a4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:41.393153 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e647a8d-abe7-4754-a8ac-b0f230163bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:41.393786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e647a8d-abe7-4754-a8ac-b0f230163bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:41.394063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e647a8d-abe7-4754-a8ac-b0f230163bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:41.394333 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e647a8d-abe7-4754-a8ac-b0f230163bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:41.395927 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8e647a8d-abe7-4754-a8ac-b0f230163bf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:41.467826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:41.501826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e647a8d-abe7-4754-a8ac-b0f230163bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:41.503295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e647a8d-abe7-4754-a8ac-b0f230163bf5 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:41.521895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 742/1484] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:43.584894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aabd0-9a8f-47ee-b6fd-10850356d53e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:43.585480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aabd0-9a8f-47ee-b6fd-10850356d53e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:43.585480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aabd0-9a8f-47ee-b6fd-10850356d53e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:43.586785 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 743/1485] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:43.592389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98eabb8-e0a4-4e1b-8baa-c636d61892cd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:43.592815 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98eabb8-e0a4-4e1b-8baa-c636d61892cd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:43.593071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98eabb8-e0a4-4e1b-8baa-c636d61892cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:43.656611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f98eabb8-e0a4-4e1b-8baa-c636d61892cd None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:43.658294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f98eabb8-e0a4-4e1b-8baa-c636d61892cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:43.755079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 743/1486] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:43.793384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99aabd0-9a8f-47ee-b6fd-10850356d53e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:43.794933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b99aabd0-9a8f-47ee-b6fd-10850356d53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:43.898674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99aabd0-9a8f-47ee-b6fd-10850356d53e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:43.969650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:43.971598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c651409-a5af-402b-90d0-cc54cccad387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:43.972230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c651409-a5af-402b-90d0-cc54cccad387 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:43.972508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c651409-a5af-402b-90d0-cc54cccad387 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:43.972778 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c651409-a5af-402b-90d0-cc54cccad387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:43.974391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2c651409-a5af-402b-90d0-cc54cccad387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:44.083074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c651409-a5af-402b-90d0-cc54cccad387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:44.084573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c651409-a5af-402b-90d0-cc54cccad387 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:44.102634 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 744/1487] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:46.269291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08694d39-ff0e-4e7d-a7b0-64443763cfba None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:46.269819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08694d39-ff0e-4e7d-a7b0-64443763cfba None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:46.269819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08694d39-ff0e-4e7d-a7b0-64443763cfba None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:46.271117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 744/1488] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:46.276343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7edc3-2128-4c2e-9a07-645472c9fcd2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:46.276782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7edc3-2128-4c2e-9a07-645472c9fcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:46.276910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7edc3-2128-4c2e-9a07-645472c9fcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:46.341576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b3f7edc3-2128-4c2e-9a07-645472c9fcd2 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:46.342781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b3f7edc3-2128-4c2e-9a07-645472c9fcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:46.394236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:46.437519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 745/1489] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:46.478515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08694d39-ff0e-4e7d-a7b0-64443763cfba None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:46.480058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-08694d39-ff0e-4e7d-a7b0-64443763cfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:46.515741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:46.584074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08694d39-ff0e-4e7d-a7b0-64443763cfba None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:46.659189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a92327-ffd5-4725-a912-7c93e9e5308b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:46.659793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a92327-ffd5-4725-a912-7c93e9e5308b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:46.660098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a92327-ffd5-4725-a912-7c93e9e5308b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:46.660372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a92327-ffd5-4725-a912-7c93e9e5308b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:46.662042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1a92327-ffd5-4725-a912-7c93e9e5308b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:46.745737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:46.774205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a92327-ffd5-4725-a912-7c93e9e5308b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:46.775644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a92327-ffd5-4725-a912-7c93e9e5308b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:46.794224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 745/1490] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:49.187478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e84cb-872b-4f52-a5d9-7558f125edae None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:49.188042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e84cb-872b-4f52-a5d9-7558f125edae None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:49.188160 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e84cb-872b-4f52-a5d9-7558f125edae None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:49.189712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 746/1491] 10.222.0.64 () {54 vars in 927 bytes} [Thu Aug 4 07:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:36:49.194813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6abb0c-2a56-46ef-9309-c5b023647c91 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:49.195223 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6abb0c-2a56-46ef-9309-c5b023647c91 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:49.195352 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6abb0c-2a56-46ef-9309-c5b023647c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:49.259778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6f6abb0c-2a56-46ef-9309-c5b023647c91 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:36:49.260924 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6f6abb0c-2a56-46ef-9309-c5b023647c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:49.355160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 746/1492] 10.222.0.64 () {58 vars in 1024 bytes} [Thu Aug 4 07:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:36:49.395706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e84cb-872b-4f52-a5d9-7558f125edae None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:36:49.397335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-748e84cb-872b-4f52-a5d9-7558f125edae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:49.499645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e84cb-872b-4f52-a5d9-7558f125edae None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:36:49.572042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1b7ed-5619-4f82-9c1b-e6e9def23aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:36:49.572590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b7ed-5619-4f82-9c1b-e6e9def23aec service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:36:49.572871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b7ed-5619-4f82-9c1b-e6e9def23aec service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:36:49.573147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b7ed-5619-4f82-9c1b-e6e9def23aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:36:49.574762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0cc1b7ed-5619-4f82-9c1b-e6e9def23aec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:36:49.584411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:49.683488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1b7ed-5619-4f82-9c1b-e6e9def23aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:36:49.684964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1b7ed-5619-4f82-9c1b-e6e9def23aec service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:36:49.703188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 747/1493] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:36:59.353457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=82 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:36:59.700785 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=50 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:08.640007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74952f-5361-4892-b924-f6028eda6ada None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:08.640603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74952f-5361-4892-b924-f6028eda6ada None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:08.640714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74952f-5361-4892-b924-f6028eda6ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:08.714291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ec74952f-5361-4892-b924-f6028eda6ada None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:38:08.715491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ec74952f-5361-4892-b924-f6028eda6ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:08.809033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 747/1494] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:38:08.817250 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad74e95-4e44-426c-9138-535f1807fe33 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:08.817750 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad74e95-4e44-426c-9138-535f1807fe33 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:08.817857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad74e95-4e44-426c-9138-535f1807fe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:08.894314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dad74e95-4e44-426c-9138-535f1807fe33 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:38:08.895450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dad74e95-4e44-426c-9138-535f1807fe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:08.995736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 748/1495] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:38:09.001094 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74952f-5361-4892-b924-f6028eda6ada None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:38:09.002305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ec74952f-5361-4892-b924-f6028eda6ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:09.013552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74952f-5361-4892-b924-f6028eda6ada None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:38:09.075218 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:09.109475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1ad89-b042-423d-bd19-b40dc2c4519b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:38:09.110085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1ad89-b042-423d-bd19-b40dc2c4519b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:09.110365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1ad89-b042-423d-bd19-b40dc2c4519b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:09.110637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1ad89-b042-423d-bd19-b40dc2c4519b admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:09.111325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1ad89-b042-423d-bd19-b40dc2c4519b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:38:09.111624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1ad89-b042-423d-bd19-b40dc2c4519b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:38:09.112673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-0be1ad89-b042-423d-bd19-b40dc2c4519b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:38:09.112673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Aug 04 07:38:09.112673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: Aug 04 07:38:09.113552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1ad89-b042-423d-bd19-b40dc2c4519b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:38:09.125437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 748/1496] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:38:09.133781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad74e95-4e44-426c-9138-535f1807fe33 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:38:09.143614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad74e95-4e44-426c-9138-535f1807fe33 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:38:09.203363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:09.236732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e31c9f-8293-4ad4-a067-319900fe880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:38:09.237345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e31c9f-8293-4ad4-a067-319900fe880a admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:09.237632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e31c9f-8293-4ad4-a067-319900fe880a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:09.237906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e31c9f-8293-4ad4-a067-319900fe880a admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:09.238585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e31c9f-8293-4ad4-a067-319900fe880a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:38:09.240341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-67e31c9f-8293-4ad4-a067-319900fe880a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:38:09.240341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Aug 04 07:38:09.240341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: Aug 04 07:38:09.241531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e31c9f-8293-4ad4-a067-319900fe880a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:38:09.343300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 749/1497] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:38:09 2022] POST /identity/v3/projects => generated 379 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:38:09.347528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1ad89-b042-423d-bd19-b40dc2c4519b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:38:09.349099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0be1ad89-b042-423d-bd19-b40dc2c4519b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:09.382929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1ad89-b042-423d-bd19-b40dc2c4519b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:38:09.474361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96836410-fff1-4a14-b7f1-ceaa9eb98adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:38:09.474896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96836410-fff1-4a14-b7f1-ceaa9eb98adc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:09.475177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96836410-fff1-4a14-b7f1-ceaa9eb98adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:09.475446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96836410-fff1-4a14-b7f1-ceaa9eb98adc admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:09.476133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96836410-fff1-4a14-b7f1-ceaa9eb98adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:38:09.477177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-96836410-fff1-4a14-b7f1-ceaa9eb98adc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:38:09.477177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Aug 04 07:38:09.477177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: Aug 04 07:38:09.478368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96836410-fff1-4a14-b7f1-ceaa9eb98adc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:38:09.577109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 749/1498] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:38:09 2022] POST /identity/v3/users => generated 290 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:38:09.581259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e31c9f-8293-4ad4-a067-319900fe880a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:38:09.582842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-67e31c9f-8293-4ad4-a067-319900fe880a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:09.644467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e31c9f-8293-4ad4-a067-319900fe880a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:38:09.728642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:09.739892 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc7dca-5f8c-4c60-857b-5069fda68852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:38:09.740611 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc7dca-5f8c-4c60-857b-5069fda68852 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:09.742316 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc7dca-5f8c-4c60-857b-5069fda68852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:09.742316 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc7dca-5f8c-4c60-857b-5069fda68852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:09.742316 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc7dca-5f8c-4c60-857b-5069fda68852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:38:09.742929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-a4fc7dca-5f8c-4c60-857b-5069fda68852 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:38:09.742929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Aug 04 07:38:09.742929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: Aug 04 07:38:09.743815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc7dca-5f8c-4c60-857b-5069fda68852 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:38:09.757152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 750/1499] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:38:09.760584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96836410-fff1-4a14-b7f1-ceaa9eb98adc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:38:09.762166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-96836410-fff1-4a14-b7f1-ceaa9eb98adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:09.773732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96836410-fff1-4a14-b7f1-ceaa9eb98adc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:38:09.799081 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:09.867623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053f5b1-68b8-4ce7-a35b-c91719905c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:38:09.868264 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053f5b1-68b8-4ce7-a35b-c91719905c9c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:09.868546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053f5b1-68b8-4ce7-a35b-c91719905c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:09.868818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053f5b1-68b8-4ce7-a35b-c91719905c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:09.869513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053f5b1-68b8-4ce7-a35b-c91719905c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:38:09.870521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-5053f5b1-68b8-4ce7-a35b-c91719905c9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:38:09.870521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Aug 04 07:38:09.870521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: Aug 04 07:38:09.871399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053f5b1-68b8-4ce7-a35b-c91719905c9c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:38:09.884668 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 750/1500] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:38:09.888852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc7dca-5f8c-4c60-857b-5069fda68852 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:38:09.891195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a4fc7dca-5f8c-4c60-857b-5069fda68852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:09.925266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc7dca-5f8c-4c60-857b-5069fda68852 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:38:10.019466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7023e-2eb1-47c1-8a3b-a14f89cbe18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:38:10.020100 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7023e-2eb1-47c1-8a3b-a14f89cbe18f admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:10.020387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7023e-2eb1-47c1-8a3b-a14f89cbe18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:10.020666 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7023e-2eb1-47c1-8a3b-a14f89cbe18f admin admin] PATH_INFO: `/v3/projects/b331b68684fc4979913351eb5837f5ab/users/ff9536ab3c134ecd9b8940825f55ab02/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:10.042202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7023e-2eb1-47c1-8a3b-a14f89cbe18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b331b68684fc4979913351eb5837f5ab, user_id=ff9536ab3c134ecd9b8940825f55ab02, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:38:10.043251 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-3cf7023e-2eb1-47c1-8a3b-a14f89cbe18f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:38:10.043251 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Aug 04 07:38:10.043251 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: Aug 04 07:38:10.045248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7023e-2eb1-47c1-8a3b-a14f89cbe18f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:38:10.127829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 751/1501] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:38:09 2022] PUT /identity/v3/projects/b331b68684fc4979913351eb5837f5ab/users/ff9536ab3c134ecd9b8940825f55ab02/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:38:10.140481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a478d3-1c2d-4a24-b968-e771d15ec1f9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:10.140777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a478d3-1c2d-4a24-b968-e771d15ec1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:10.141240 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a478d3-1c2d-4a24-b968-e771d15ec1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:10.198263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b1a478d3-1c2d-4a24-b968-e771d15ec1f9 None None] MFA Rules not processed for user `ff9536ab3c134ecd9b8940825f55ab02`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:38:10.199435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b1a478d3-1c2d-4a24-b968-e771d15ec1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:10.277332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 751/1502] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:38:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:38:10.290487 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a8c27-f2ed-48cd-8053-96274a8088d8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:10.290781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a8c27-f2ed-48cd-8053-96274a8088d8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:10.291066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a8c27-f2ed-48cd-8053-96274a8088d8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:10.292457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 752/1503] 10.222.0.64 () {54 vars in 964 bytes} [Thu Aug 4 07:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:38:10.297920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74856fa-6939-48e4-917a-afeccb9b67e3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:10.298211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74856fa-6939-48e4-917a-afeccb9b67e3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:10.298492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74856fa-6939-48e4-917a-afeccb9b67e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:10.329756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:10.374098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a74856fa-6939-48e4-917a-afeccb9b67e3 None None] MFA Rules not processed for user `eab096d4ccae4bc09e6177afd85a55de`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:38:10.375417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a74856fa-6939-48e4-917a-afeccb9b67e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:10.448693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 752/1504] 10.222.0.64 () {58 vars in 1061 bytes} [Thu Aug 4 07:38:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:38:10.456581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a8c27-f2ed-48cd-8053-96274a8088d8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:38:10.458106 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c72a8c27-f2ed-48cd-8053-96274a8088d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:10.507772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:10.593723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a8c27-f2ed-48cd-8053-96274a8088d8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:38:10.635323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c810748-c443-49c8-9c85-ad898cd5be86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:38:10.635941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c810748-c443-49c8-9c85-ad898cd5be86 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:10.636255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c810748-c443-49c8-9c85-ad898cd5be86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:10.636536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c810748-c443-49c8-9c85-ad898cd5be86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:10.638167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0c810748-c443-49c8-9c85-ad898cd5be86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:10.737526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c810748-c443-49c8-9c85-ad898cd5be86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:38:10.744102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c810748-c443-49c8-9c85-ad898cd5be86 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:38:10.744102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:10.756860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 753/1505] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:38:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:38:10.808823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71d50-9bda-441b-b5d7-52c83c03e13f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:10.809349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71d50-9bda-441b-b5d7-52c83c03e13f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:10.809457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71d50-9bda-441b-b5d7-52c83c03e13f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:10.810724 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 753/1506] 10.222.0.64 () {52 vars in 830 bytes} [Thu Aug 4 07:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:38:10.814217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c810748-c443-49c8-9c85-ad898cd5be86 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:38:10.815418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0c810748-c443-49c8-9c85-ad898cd5be86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:10.917513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c810748-c443-49c8-9c85-ad898cd5be86 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:38:11.010315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:38:11.010957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:11.011242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:11.011518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] PATH_INFO: `/v3/users/ff9536ab3c134ecd9b8940825f55ab02` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:11.024334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff9536ab3c134ecd9b8940825f55ab02)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:38:11.025365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:38:11.025365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Aug 04 07:38:11.025365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: Aug 04 07:38:11.026626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:38:11.037007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:11.242747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff9536ab3c134ecd9b8940825f55ab02'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:38:11.243279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff9536ab3c134ecd9b8940825f55ab02'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:38:11.243698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff9536ab3c134ecd9b8940825f55ab02'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:38:11.266123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff9536ab3c134ecd9b8940825f55ab02'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:38:11.352872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff9536ab3c134ecd9b8940825f55ab02'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:38:11.373173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 754/1507] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:38:10 2022] DELETE /identity/v3/users/ff9536ab3c134ecd9b8940825f55ab02 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:38:11.377718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d68d00-223f-4ba2-91f7-aecf02e36769 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:11.378012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d68d00-223f-4ba2-91f7-aecf02e36769 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:11.378327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d68d00-223f-4ba2-91f7-aecf02e36769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:11.434435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-10d68d00-223f-4ba2-91f7-aecf02e36769 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:38:11.435639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-10d68d00-223f-4ba2-91f7-aecf02e36769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:11.509811 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:11.522963 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 754/1508] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:38:11.533752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:38:11.535033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:11.634345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f244ad-9fd7-4452-a990-0f15186c3cb5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:38:11.687805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:11.707065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba7987-3d8f-43e9-b056-82646865546b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:38:11.707687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba7987-3d8f-43e9-b056-82646865546b service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:11.707973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba7987-3d8f-43e9-b056-82646865546b service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:11.708278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba7987-3d8f-43e9-b056-82646865546b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:11.709897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6dba7987-3d8f-43e9-b056-82646865546b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:11.821050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba7987-3d8f-43e9-b056-82646865546b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:38:11.822653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba7987-3d8f-43e9-b056-82646865546b service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:38:11.841919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 755/1509] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:38:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:38:12.282266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d68d00-223f-4ba2-91f7-aecf02e36769 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:38:12.283708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-10d68d00-223f-4ba2-91f7-aecf02e36769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:38:12.294180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d68d00-223f-4ba2-91f7-aecf02e36769 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:38:12.397661 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:38:12.398358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:38:12.398636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:38:12.398907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] PATH_INFO: `/v3/projects/b331b68684fc4979913351eb5837f5ab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:38:12.409039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b331b68684fc4979913351eb5837f5ab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:38:12.410323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:38:12.410323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Aug 04 07:38:12.410323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: Aug 04 07:38:12.411506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:38:12.514144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b331b68684fc4979913351eb5837f5ab', 'request_id': 'req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:38:12.570109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b331b68684fc4979913351eb5837f5ab', 'request_id': 'req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:38:12.576401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b331b68684fc4979913351eb5837f5ab', 'request_id': 'req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:38:12.606395 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] The token cache is being invalidate because project b331b68684fc4979913351eb5837f5ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:38:12.606680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:38:12.607102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0fa47132-a9e8-4223-b7a0-8b7716dc04f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:38:12.609475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 755/1510] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:38:12 2022] DELETE /identity/v3/projects/b331b68684fc4979913351eb5837f5ab => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:38:21.839638 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=62 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:38:22.604450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:39:40.713272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d614d5-9e56-4c08-906c-40bbc8224ed0 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:40.713846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d614d5-9e56-4c08-906c-40bbc8224ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:40.713846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d614d5-9e56-4c08-906c-40bbc8224ed0 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:40.715225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:39:40.720001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1463bf7-3de2-4e95-8470-7848c374c5a2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:40.720447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1463bf7-3de2-4e95-8470-7848c374c5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:40.720588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1463bf7-3de2-4e95-8470-7848c374c5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:40.802924 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d1463bf7-3de2-4e95-8470-7848c374c5a2 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:39:40.804164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1463bf7-3de2-4e95-8470-7848c374c5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:39:40.906486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:39:41.062041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d614d5-9e56-4c08-906c-40bbc8224ed0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:39:41.064218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-59d614d5-9e56-4c08-906c-40bbc8224ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:39:41.183100 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d614d5-9e56-4c08-906c-40bbc8224ed0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:39:41.188596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640d21e-d4e0-4e16-876c-5112a6bd292c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:39:41.189209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640d21e-d4e0-4e16-876c-5112a6bd292c service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:41.189488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640d21e-d4e0-4e16-876c-5112a6bd292c service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:41.189761 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640d21e-d4e0-4e16-876c-5112a6bd292c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:41.197998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640d21e-d4e0-4e16-876c-5112a6bd292c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:39:41.199576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640d21e-d4e0-4e16-876c-5112a6bd292c service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:39:41.208962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 757/1513] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:39:42.054862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325999f1-e0b1-4626-8e0d-d0daf3249e1a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:42.055390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325999f1-e0b1-4626-8e0d-d0daf3249e1a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:42.055390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325999f1-e0b1-4626-8e0d-d0daf3249e1a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:42.056808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:39:42.061475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed19b5-b630-4385-9674-2ef600b069f8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:42.061874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed19b5-b630-4385-9674-2ef600b069f8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:42.062025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed19b5-b630-4385-9674-2ef600b069f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:42.082024 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-28ed19b5-b630-4385-9674-2ef600b069f8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:39:42.083237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-28ed19b5-b630-4385-9674-2ef600b069f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:39:42.091348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:39:42.245246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325999f1-e0b1-4626-8e0d-d0daf3249e1a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:39:42.260714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325999f1-e0b1-4626-8e0d-d0daf3249e1a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:39:42.266349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec70d3-a6bb-4bd0-9cc9-6576dbd3a23e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:39:42.266961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec70d3-a6bb-4bd0-9cc9-6576dbd3a23e service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:42.267255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec70d3-a6bb-4bd0-9cc9-6576dbd3a23e service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:42.267554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec70d3-a6bb-4bd0-9cc9-6576dbd3a23e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:42.276847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec70d3-a6bb-4bd0-9cc9-6576dbd3a23e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:39:42.278511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec70d3-a6bb-4bd0-9cc9-6576dbd3a23e service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:39:42.288981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 758/1516] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:39:42.550879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280190b0-aad6-458f-9c34-1ec31929259b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:42.551097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280190b0-aad6-458f-9c34-1ec31929259b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:42.551382 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280190b0-aad6-458f-9c34-1ec31929259b None None] PATH_INFO: `/v3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:42.552702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 759/1517] 10.222.0.45 () {54 vars in 925 bytes} [Thu Aug 4 07:39:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:39:42.558955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb5c00-b26d-4e90-8044-0766ef622137 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:42.559247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb5c00-b26d-4e90-8044-0766ef622137 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:42.559528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb5c00-b26d-4e90-8044-0766ef622137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:42.618232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-21cb5c00-b26d-4e90-8044-0766ef622137 None None] MFA Rules not processed for user `affaeff3cd294584b7238909e0457f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:39:42.619437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21cb5c00-b26d-4e90-8044-0766ef622137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:39:42.636269 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:39:42.689753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 759/1518] 10.222.0.45 () {58 vars in 1015 bytes} [Thu Aug 4 07:39:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:39:46.602564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d667e5a-79f7-4a30-bf2f-8804d403e557 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:46.603087 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d667e5a-79f7-4a30-bf2f-8804d403e557 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:46.603087 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d667e5a-79f7-4a30-bf2f-8804d403e557 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:46.604400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 760/1519] 10.222.0.64 () {54 vars in 961 bytes} [Thu Aug 4 07:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:39:46.609307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cb6c3-b35c-4578-8df3-ece3717643a1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:46.609726 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cb6c3-b35c-4578-8df3-ece3717643a1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:46.609898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cb6c3-b35c-4578-8df3-ece3717643a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:46.631497 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-286cb6c3-b35c-4578-8df3-ece3717643a1 None None] MFA Rules not processed for user `affaeff3cd294584b7238909e0457f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:39:46.632733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-286cb6c3-b35c-4578-8df3-ece3717643a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:39:46.641301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 760/1520] 10.222.0.64 () {58 vars in 1058 bytes} [Thu Aug 4 07:39:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:39:46.648871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d667e5a-79f7-4a30-bf2f-8804d403e557 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:39:46.661810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d667e5a-79f7-4a30-bf2f-8804d403e557 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:39:46.666735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca471c9-1d2c-454e-a151-68dea82a07f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:39:46.667313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca471c9-1d2c-454e-a151-68dea82a07f7 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:46.667594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca471c9-1d2c-454e-a151-68dea82a07f7 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:46.667868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca471c9-1d2c-454e-a151-68dea82a07f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:46.676494 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca471c9-1d2c-454e-a151-68dea82a07f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:39:46.678056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca471c9-1d2c-454e-a151-68dea82a07f7 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:39:46.687417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 761/1521] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:39:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:39:53.531802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55744a9-331e-4ea2-932e-7a8f313375dc None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:53.532374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55744a9-331e-4ea2-932e-7a8f313375dc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:53.532374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55744a9-331e-4ea2-932e-7a8f313375dc None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:53.533718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:39:53.538473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067baf2-b41e-4ccc-bac5-616c4a62bfe0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:53.538919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067baf2-b41e-4ccc-bac5-616c4a62bfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:53.539050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067baf2-b41e-4ccc-bac5-616c4a62bfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:53.558741 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2067baf2-b41e-4ccc-bac5-616c4a62bfe0 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:39:53.559931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2067baf2-b41e-4ccc-bac5-616c4a62bfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:39:53.568832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:39:53.723482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55744a9-331e-4ea2-932e-7a8f313375dc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:39:53.738565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55744a9-331e-4ea2-932e-7a8f313375dc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:39:53.744158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7c3b0-03d5-4a18-b2c2-b9ef2b7bdbbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:39:53.744775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7c3b0-03d5-4a18-b2c2-b9ef2b7bdbbf service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:53.745061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7c3b0-03d5-4a18-b2c2-b9ef2b7bdbbf service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:53.745340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7c3b0-03d5-4a18-b2c2-b9ef2b7bdbbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:53.754271 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7c3b0-03d5-4a18-b2c2-b9ef2b7bdbbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:39:53.755857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7c3b0-03d5-4a18-b2c2-b9ef2b7bdbbf service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:39:53.765221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 762/1524] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:39:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:39:54.602722 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952c84f-a9be-45ca-bff5-33cc51b4bc34 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:54.603400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952c84f-a9be-45ca-bff5-33cc51b4bc34 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:54.603400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952c84f-a9be-45ca-bff5-33cc51b4bc34 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:54.604889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 07:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:39:54.610472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5214-078b-42be-a8ec-c62cc248c121 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:54.610938 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5214-078b-42be-a8ec-c62cc248c121 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:54.611059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5214-078b-42be-a8ec-c62cc248c121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:54.634414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dd8c5214-078b-42be-a8ec-c62cc248c121 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:39:54.635616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dd8c5214-078b-42be-a8ec-c62cc248c121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:39:54.643766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 07:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:39:54.812050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952c84f-a9be-45ca-bff5-33cc51b4bc34 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:39:54.826555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952c84f-a9be-45ca-bff5-33cc51b4bc34 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:39:54.832711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c8f45-53e4-442d-9d52-5c98d045de51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:39:54.833324 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c8f45-53e4-442d-9d52-5c98d045de51 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:54.833612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c8f45-53e4-442d-9d52-5c98d045de51 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:54.833891 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c8f45-53e4-442d-9d52-5c98d045de51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:54.842353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c8f45-53e4-442d-9d52-5c98d045de51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:39:54.843964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c8f45-53e4-442d-9d52-5c98d045de51 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:39:54.854195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 764/1527] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:39:59.168193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818cae8-5aa0-424d-b468-0c62a7bd90fe None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:59.168750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818cae8-5aa0-424d-b468-0c62a7bd90fe None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:59.168750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818cae8-5aa0-424d-b468-0c62a7bd90fe None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:59.170150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:39:59.175464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5900b47-4623-46b4-a96f-484d35f18477 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:59.175902 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5900b47-4623-46b4-a96f-484d35f18477 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:59.176105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5900b47-4623-46b4-a96f-484d35f18477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:59.195683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e5900b47-4623-46b4-a96f-484d35f18477 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:39:59.196921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e5900b47-4623-46b4-a96f-484d35f18477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:39:59.204775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:39:59.278359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818cae8-5aa0-424d-b468-0c62a7bd90fe None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:39:59.280073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9818cae8-5aa0-424d-b468-0c62a7bd90fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:39:59.356773 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818cae8-5aa0-424d-b468-0c62a7bd90fe None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:39:59.360298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03b7da-668d-4b46-886c-579280adac26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:39:59.360909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03b7da-668d-4b46-886c-579280adac26 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:39:59.361191 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03b7da-668d-4b46-886c-579280adac26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:39:59.361466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03b7da-668d-4b46-886c-579280adac26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:39:59.370787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03b7da-668d-4b46-886c-579280adac26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:39:59.372245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03b7da-668d-4b46-886c-579280adac26 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:39:59.381895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 765/1530] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:40:01.366652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bea76e-38e4-4cf7-98e1-fd4f436e0d56 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:40:01.367194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bea76e-38e4-4cf7-98e1-fd4f436e0d56 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:40:01.367194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bea76e-38e4-4cf7-98e1-fd4f436e0d56 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:40:01.368504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:40:01.373679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25c42f-de46-493f-a9f0-d3d8445db09e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:40:01.374080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25c42f-de46-493f-a9f0-d3d8445db09e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:40:01.374244 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25c42f-de46-493f-a9f0-d3d8445db09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:40:01.393359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3d25c42f-de46-493f-a9f0-d3d8445db09e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:40:01.394648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3d25c42f-de46-493f-a9f0-d3d8445db09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:40:01.402369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:40:01.438571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bea76e-38e4-4cf7-98e1-fd4f436e0d56 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:40:01.449218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bea76e-38e4-4cf7-98e1-fd4f436e0d56 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:40:01.452758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78eadb6-469c-47d8-bf30-9b4fbc7fd6a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:40:01.453360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78eadb6-469c-47d8-bf30-9b4fbc7fd6a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:40:01.453644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78eadb6-469c-47d8-bf30-9b4fbc7fd6a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:40:01.453922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78eadb6-469c-47d8-bf30-9b4fbc7fd6a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:40:01.463113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78eadb6-469c-47d8-bf30-9b4fbc7fd6a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:40:01.464578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78eadb6-469c-47d8-bf30-9b4fbc7fd6a6 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:40:01.474484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 767/1533] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:40:05.237062 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f5be0-dd03-46a7-871a-904fd82d1c5b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:40:05.238239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f5be0-dd03-46a7-871a-904fd82d1c5b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:40:05.238239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f5be0-dd03-46a7-871a-904fd82d1c5b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:40:05.239425 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:40:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:40:05.244525 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e12a46-0b75-4001-98a1-1be6a7ec2f92 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:40:05.244976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e12a46-0b75-4001-98a1-1be6a7ec2f92 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:40:05.245093 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e12a46-0b75-4001-98a1-1be6a7ec2f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:40:05.252679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=81 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:40:05.264626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-20e12a46-0b75-4001-98a1-1be6a7ec2f92 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:40:05.275961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-20e12a46-0b75-4001-98a1-1be6a7ec2f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:40:05.275961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:40:05.577634 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f5be0-dd03-46a7-871a-904fd82d1c5b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:40:05.579364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b66f5be0-dd03-46a7-871a-904fd82d1c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:40:05.645756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:40:05.659835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f5be0-dd03-46a7-871a-904fd82d1c5b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:40:05.663593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20879277-9fc2-4c9e-8a3d-06f0c16a2c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:40:05.664234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20879277-9fc2-4c9e-8a3d-06f0c16a2c41 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:40:05.664514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20879277-9fc2-4c9e-8a3d-06f0c16a2c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:40:05.664785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20879277-9fc2-4c9e-8a3d-06f0c16a2c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:40:05.673803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20879277-9fc2-4c9e-8a3d-06f0c16a2c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:40:05.675848 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20879277-9fc2-4c9e-8a3d-06f0c16a2c41 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:40:05.685402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 768/1536] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:40:07.699734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdccb0-e9f7-4f03-90c7-1eb0a6bcaa55 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:40:07.700361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdccb0-e9f7-4f03-90c7-1eb0a6bcaa55 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:40:07.700361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdccb0-e9f7-4f03-90c7-1eb0a6bcaa55 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:40:07.701706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 07:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:40:07.707363 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b986580-4c5e-4e92-9a53-fd1afe0e288b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:40:07.707772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b986580-4c5e-4e92-9a53-fd1afe0e288b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:40:07.707950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b986580-4c5e-4e92-9a53-fd1afe0e288b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:40:07.727775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3b986580-4c5e-4e92-9a53-fd1afe0e288b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:40:07.729001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3b986580-4c5e-4e92-9a53-fd1afe0e288b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:40:07.736925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 07:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:40:08.038221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdccb0-e9f7-4f03-90c7-1eb0a6bcaa55 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:40:08.039958 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2ccdccb0-e9f7-4f03-90c7-1eb0a6bcaa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:40:08.053674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdccb0-e9f7-4f03-90c7-1eb0a6bcaa55 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:40:08.057122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70756afc-8c81-4f04-85bf-2bd087d31355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:40:08.057757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70756afc-8c81-4f04-85bf-2bd087d31355 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:40:08.058043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70756afc-8c81-4f04-85bf-2bd087d31355 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:40:08.058319 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70756afc-8c81-4f04-85bf-2bd087d31355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:40:08.067092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70756afc-8c81-4f04-85bf-2bd087d31355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:40:08.068564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70756afc-8c81-4f04-85bf-2bd087d31355 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:40:08.077956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 770/1539] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:40:17.723190 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=11 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:40:18.073836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=8 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:31.425617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c92b-6fa5-4ab9-a4cd-4cda39a00d8e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:31.426241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c92b-6fa5-4ab9-a4cd-4cda39a00d8e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:31.426241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c92b-6fa5-4ab9-a4cd-4cda39a00d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:31.449458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7bf1c92b-6fa5-4ab9-a4cd-4cda39a00d8e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:31.451017 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7bf1c92b-6fa5-4ab9-a4cd-4cda39a00d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:31.464291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 770/1540] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:31.474211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b7b2-3dce-4ef3-98f2-fff34aa91e6c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:31.476069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b7b2-3dce-4ef3-98f2-fff34aa91e6c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:31.476069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b7b2-3dce-4ef3-98f2-fff34aa91e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:31.506480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2957b7b2-3dce-4ef3-98f2-fff34aa91e6c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:31.507733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2957b7b2-3dce-4ef3-98f2-fff34aa91e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:31.522521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 771/1541] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:31.527792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1c92b-6fa5-4ab9-a4cd-4cda39a00d8e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:31.544296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1c92b-6fa5-4ab9-a4cd-4cda39a00d8e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:31.552826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94029b-1146-4045-8a3d-6a529576d1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:31.553727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94029b-1146-4045-8a3d-6a529576d1cc admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:31.554191 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94029b-1146-4045-8a3d-6a529576d1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:31.554653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94029b-1146-4045-8a3d-6a529576d1cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:31.555524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94029b-1146-4045-8a3d-6a529576d1cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:31.556019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94029b-1146-4045-8a3d-6a529576d1cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:31.557801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94029b-1146-4045-8a3d-6a529576d1cc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:31.576162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 771/1542] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:41:31.585863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957b7b2-3dce-4ef3-98f2-fff34aa91e6c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:31.596555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957b7b2-3dce-4ef3-98f2-fff34aa91e6c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:31.610846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef1ed4-4e26-46bd-8556-ec017e20312d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:31.611547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef1ed4-4e26-46bd-8556-ec017e20312d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:31.611836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef1ed4-4e26-46bd-8556-ec017e20312d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:31.612145 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef1ed4-4e26-46bd-8556-ec017e20312d admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:31.613032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef1ed4-4e26-46bd-8556-ec017e20312d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:31.614958 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef1ed4-4e26-46bd-8556-ec017e20312d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:31.660242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8246385-6432-460b-b95d-d34009cb71ba None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:31.660925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8246385-6432-460b-b95d-d34009cb71ba None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:31.661379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8246385-6432-460b-b95d-d34009cb71ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:31.695690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b8246385-6432-460b-b95d-d34009cb71ba None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:31.697192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b8246385-6432-460b-b95d-d34009cb71ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:31.709760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 772/1543] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:41:31.719072 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f9aac-a280-46b0-a942-4f3c04c1f1de None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:31.719616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f9aac-a280-46b0-a942-4f3c04c1f1de None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:31.720147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f9aac-a280-46b0-a942-4f3c04c1f1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:31.752607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1e3f9aac-a280-46b0-a942-4f3c04c1f1de None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:31.754267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1e3f9aac-a280-46b0-a942-4f3c04c1f1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:31.756630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 772/1544] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:31 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:31.770928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef1ed4-4e26-46bd-8556-ec017e20312d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:31.850493 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 773/1545] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:41:31.863893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f9aac-a280-46b0-a942-4f3c04c1f1de None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:31.872146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f9aac-a280-46b0-a942-4f3c04c1f1de None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:31.884703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780437b-bff2-414f-a067-ee4f789e2a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:31.885352 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780437b-bff2-414f-a067-ee4f789e2a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:31.885643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780437b-bff2-414f-a067-ee4f789e2a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:31.885926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780437b-bff2-414f-a067-ee4f789e2a20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:31.886811 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780437b-bff2-414f-a067-ee4f789e2a20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:31.887136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780437b-bff2-414f-a067-ee4f789e2a20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:31.888865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780437b-bff2-414f-a067-ee4f789e2a20 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:31.891078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef1ed4-4e26-46bd-8556-ec017e20312d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:31.905643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b959ef-339f-42a4-bab6-489a2233f4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:31.906661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b959ef-339f-42a4-bab6-489a2233f4f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:31.907154 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b959ef-339f-42a4-bab6-489a2233f4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:31.907650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b959ef-339f-42a4-bab6-489a2233f4f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:31.908631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b959ef-339f-42a4-bab6-489a2233f4f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:31.909928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-71b959ef-339f-42a4-bab6-489a2233f4f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:41:31.909928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Aug 04 07:41:31.909928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: Aug 04 07:41:31.911649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b959ef-339f-42a4-bab6-489a2233f4f0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:31.915068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 774/1546] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:41:31.916514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f428c4-3484-438f-b694-b5d20680baee None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:31.916813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f428c4-3484-438f-b694-b5d20680baee None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:31.917125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f428c4-3484-438f-b694-b5d20680baee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:31.955957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f3f428c4-3484-438f-b694-b5d20680baee None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:31.957553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f3f428c4-3484-438f-b694-b5d20680baee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:31.972210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 775/1547] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:31.973351 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f428c4-3484-438f-b694-b5d20680baee None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:31.994750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f428c4-3484-438f-b694-b5d20680baee None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.004028 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1814b7-5bbc-47a8-b7f2-ec5f7e288f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.005226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1814b7-5bbc-47a8-b7f2-ec5f7e288f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.005708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1814b7-5bbc-47a8-b7f2-ec5f7e288f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.006175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1814b7-5bbc-47a8-b7f2-ec5f7e288f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.007064 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1814b7-5bbc-47a8-b7f2-ec5f7e288f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.008344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-5d1814b7-5bbc-47a8-b7f2-ec5f7e288f55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:41:32.008344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Aug 04 07:41:32.008344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: Aug 04 07:41:32.010045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1814b7-5bbc-47a8-b7f2-ec5f7e288f55 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.018158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 773/1548] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:31 2022] POST /identity/v3/users => generated 288 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:41:32.018158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df01c2f-9722-4725-836c-439b2c6a7bf2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.018158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df01c2f-9722-4725-836c-439b2c6a7bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.018158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df01c2f-9722-4725-836c-439b2c6a7bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.069465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0df01c2f-9722-4725-836c-439b2c6a7bf2 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.070690 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0df01c2f-9722-4725-836c-439b2c6a7bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.085610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 774/1549] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.086451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df01c2f-9722-4725-836c-439b2c6a7bf2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.101687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df01c2f-9722-4725-836c-439b2c6a7bf2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.108915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592cf3d-818a-4ba5-a4c0-50a7dda2f9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.109575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592cf3d-818a-4ba5-a4c0-50a7dda2f9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.109863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592cf3d-818a-4ba5-a4c0-50a7dda2f9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.110144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592cf3d-818a-4ba5-a4c0-50a7dda2f9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.110868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592cf3d-818a-4ba5-a4c0-50a7dda2f9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.112816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592cf3d-818a-4ba5-a4c0-50a7dda2f9e5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.129009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 775/1550] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:32.129961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592cf3d-818a-4ba5-a4c0-50a7dda2f9e5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.144338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592cf3d-818a-4ba5-a4c0-50a7dda2f9e5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.153267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c8740-6b35-42d9-84b9-4f97606db43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.154763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c8740-6b35-42d9-84b9-4f97606db43f admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.154763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c8740-6b35-42d9-84b9-4f97606db43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.154763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c8740-6b35-42d9-84b9-4f97606db43f admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.155167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c8740-6b35-42d9-84b9-4f97606db43f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:32.155479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c8740-6b35-42d9-84b9-4f97606db43f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.160040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-d33c8740-6b35-42d9-84b9-4f97606db43f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:41:32.160040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Aug 04 07:41:32.160040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: Aug 04 07:41:32.160040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c8740-6b35-42d9-84b9-4f97606db43f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.167922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 776/1551] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:31 2022] POST /identity/v3/projects => generated 405 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:32.169569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c6904-063a-4f4b-8adf-af428a19e9ac None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.170077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c6904-063a-4f4b-8adf-af428a19e9ac None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.170587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c6904-063a-4f4b-8adf-af428a19e9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.177779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 776/1552] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:41:32.177779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c8740-6b35-42d9-84b9-4f97606db43f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.185664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c8740-6b35-42d9-84b9-4f97606db43f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.207245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3c0c6904-063a-4f4b-8adf-af428a19e9ac None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.208447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3c0c6904-063a-4f4b-8adf-af428a19e9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.260455 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babeeda-c4e5-4508-9313-77d7b7d7901e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.261090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babeeda-c4e5-4508-9313-77d7b7d7901e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.261380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babeeda-c4e5-4508-9313-77d7b7d7901e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.261659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babeeda-c4e5-4508-9313-77d7b7d7901e admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.262365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babeeda-c4e5-4508-9313-77d7b7d7901e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.263961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babeeda-c4e5-4508-9313-77d7b7d7901e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.274281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 777/1553] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:32.275335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ef51e-4aec-437c-96fa-f220267166e6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.275629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ef51e-4aec-437c-96fa-f220267166e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.275920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ef51e-4aec-437c-96fa-f220267166e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.278759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 777/1554] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.280278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c6904-063a-4f4b-8adf-af428a19e9ac None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.294368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c6904-063a-4f4b-8adf-af428a19e9ac None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.295786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3f9ef51e-4aec-437c-96fa-f220267166e6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.297005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3f9ef51e-4aec-437c-96fa-f220267166e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.301097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4268a-df95-45ec-8d60-6eae2e484a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.301986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4268a-df95-45ec-8d60-6eae2e484a10 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.302432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4268a-df95-45ec-8d60-6eae2e484a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.302867 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4268a-df95-45ec-8d60-6eae2e484a10 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.303743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4268a-df95-45ec-8d60-6eae2e484a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.305821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 778/1555] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.306486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4268a-df95-45ec-8d60-6eae2e484a10 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.306616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ef51e-4aec-437c-96fa-f220267166e6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.316939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ef51e-4aec-437c-96fa-f220267166e6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.323397 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23e77a-a40b-496c-a509-06be59fa0b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.324063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23e77a-a40b-496c-a509-06be59fa0b3c admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.326632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23e77a-a40b-496c-a509-06be59fa0b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.326632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23e77a-a40b-496c-a509-06be59fa0b3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.326632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23e77a-a40b-496c-a509-06be59fa0b3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.327740 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23e77a-a40b-496c-a509-06be59fa0b3c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.338381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:32.384710 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 778/1556] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/users => generated 303 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:32.386022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97611df0-5385-4c8f-a337-3b4917be5eae None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.386315 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97611df0-5385-4c8f-a337-3b4917be5eae None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.386592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97611df0-5385-4c8f-a337-3b4917be5eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.405904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-97611df0-5385-4c8f-a337-3b4917be5eae None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.407120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97611df0-5385-4c8f-a337-3b4917be5eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.414739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 779/1557] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:32.415110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 779/1558] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.415110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97611df0-5385-4c8f-a337-3b4917be5eae None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.422450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2067ba9-e6ce-409c-af68-f4a4c6085299 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.422450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2067ba9-e6ce-409c-af68-f4a4c6085299 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.422450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2067ba9-e6ce-409c-af68-f4a4c6085299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.429143 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97611df0-5385-4c8f-a337-3b4917be5eae None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.433969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:32.441074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a2067ba9-e6ce-409c-af68-f4a4c6085299 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.442471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a2067ba9-e6ce-409c-af68-f4a4c6085299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.486754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786a876-ca2a-49dd-9f6b-a7d3883825aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.487525 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786a876-ca2a-49dd-9f6b-a7d3883825aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.487794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786a876-ca2a-49dd-9f6b-a7d3883825aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.490590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786a876-ca2a-49dd-9f6b-a7d3883825aa admin admin] PATH_INFO: `/v3/projects/3397da2ce7ff494389f4134624f3a659/users/e44a63323cfb40a2bb7a5e83d8636f38/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.503452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786a876-ca2a-49dd-9f6b-a7d3883825aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3397da2ce7ff494389f4134624f3a659, user_id=e44a63323cfb40a2bb7a5e83d8636f38, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.504529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-4786a876-ca2a-49dd-9f6b-a7d3883825aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:41:32.504529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Aug 04 07:41:32.504529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: Aug 04 07:41:32.508197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 780/1559] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.508197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c75f3a-b582-4072-83e9-b6ba2ca0aedb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.508197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c75f3a-b582-4072-83e9-b6ba2ca0aedb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.508197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c75f3a-b582-4072-83e9-b6ba2ca0aedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.508876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786a876-ca2a-49dd-9f6b-a7d3883825aa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.528350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-03c75f3a-b582-4072-83e9-b6ba2ca0aedb None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.529577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-03c75f3a-b582-4072-83e9-b6ba2ca0aedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.538027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 781/1560] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.539189 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c28a6-a6cc-4dc1-8017-662f1aa71012 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.539492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c28a6-a6cc-4dc1-8017-662f1aa71012 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.539755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c28a6-a6cc-4dc1-8017-662f1aa71012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.558844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-772c28a6-a6cc-4dc1-8017-662f1aa71012 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.568192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-772c28a6-a6cc-4dc1-8017-662f1aa71012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.568868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 782/1561] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.570197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a3c8a-de17-4768-94c6-ab95fa29fe39 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.570485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a3c8a-de17-4768-94c6-ab95fa29fe39 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.570767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a3c8a-de17-4768-94c6-ab95fa29fe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.589623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0a0a3c8a-de17-4768-94c6-ab95fa29fe39 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.590787 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0a0a3c8a-de17-4768-94c6-ab95fa29fe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.599561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 783/1562] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.600357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a3c8a-de17-4768-94c6-ab95fa29fe39 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.606497 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 780/1563] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:32 2022] PUT /identity/v3/projects/3397da2ce7ff494389f4134624f3a659/users/e44a63323cfb40a2bb7a5e83d8636f38/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:32.607195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786a876-ca2a-49dd-9f6b-a7d3883825aa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.614406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a3c8a-de17-4768-94c6-ab95fa29fe39 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.668659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786a876-ca2a-49dd-9f6b-a7d3883825aa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.672150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6b802-bbbb-438c-a052-82b8d9d91b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.672781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6b802-bbbb-438c-a052-82b8d9d91b55 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.673067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6b802-bbbb-438c-a052-82b8d9d91b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.673346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642708d-9fa0-4e45-ac02-7c5b08d29f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.674131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6b802-bbbb-438c-a052-82b8d9d91b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.674524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642708d-9fa0-4e45-ac02-7c5b08d29f7a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.674853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6b802-bbbb-438c-a052-82b8d9d91b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.674973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642708d-9fa0-4e45-ac02-7c5b08d29f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.675185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642708d-9fa0-4e45-ac02-7c5b08d29f7a admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.676495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6b802-bbbb-438c-a052-82b8d9d91b55 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.676817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642708d-9fa0-4e45-ac02-7c5b08d29f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.678774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642708d-9fa0-4e45-ac02-7c5b08d29f7a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.686588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 784/1564] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:32.687750 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16f49f-7aae-4efa-b4b5-2dec226655d4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.688069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16f49f-7aae-4efa-b4b5-2dec226655d4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.688350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16f49f-7aae-4efa-b4b5-2dec226655d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.708197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cc16f49f-7aae-4efa-b4b5-2dec226655d4 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.709655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cc16f49f-7aae-4efa-b4b5-2dec226655d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.717563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 785/1565] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.718699 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca57c77-de8e-4671-83f9-35c213fc3b91 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.718991 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca57c77-de8e-4671-83f9-35c213fc3b91 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.719269 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca57c77-de8e-4671-83f9-35c213fc3b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.740975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9ca57c77-de8e-4671-83f9-35c213fc3b91 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.742109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9ca57c77-de8e-4671-83f9-35c213fc3b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.750259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 786/1566] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.751025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca57c77-de8e-4671-83f9-35c213fc3b91 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.760396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca57c77-de8e-4671-83f9-35c213fc3b91 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.765117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee6860-f747-473e-ba2c-1bc9e452471d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.765749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee6860-f747-473e-ba2c-1bc9e452471d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.766028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee6860-f747-473e-ba2c-1bc9e452471d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.766301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee6860-f747-473e-ba2c-1bc9e452471d admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.766993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee6860-f747-473e-ba2c-1bc9e452471d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:32.767392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee6860-f747-473e-ba2c-1bc9e452471d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.769074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee6860-f747-473e-ba2c-1bc9e452471d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.776201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 781/1567] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:32.776201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642708d-9fa0-4e45-ac02-7c5b08d29f7a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.782439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 787/1568] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:41:32.784245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9aa573-07b2-4999-9e81-9ddc7a4ce0a8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.784532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9aa573-07b2-4999-9e81-9ddc7a4ce0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.784807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9aa573-07b2-4999-9e81-9ddc7a4ce0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.792463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642708d-9fa0-4e45-ac02-7c5b08d29f7a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.797060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5a4af-41e5-47be-a2fc-cdffc95b25ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.797743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5a4af-41e5-47be-a2fc-cdffc95b25ef admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.798009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5a4af-41e5-47be-a2fc-cdffc95b25ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.798286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5a4af-41e5-47be-a2fc-cdffc95b25ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.799825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5a4af-41e5-47be-a2fc-cdffc95b25ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:32.800185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5a4af-41e5-47be-a2fc-cdffc95b25ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.801837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5a4af-41e5-47be-a2fc-cdffc95b25ef admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.807151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8e9aa573-07b2-4999-9e81-9ddc7a4ce0a8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.810483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8e9aa573-07b2-4999-9e81-9ddc7a4ce0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.815096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 782/1569] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:41:32.816577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ea9fc-f892-45bb-93ef-693437a82c48 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.816877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ea9fc-f892-45bb-93ef-693437a82c48 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.817152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ea9fc-f892-45bb-93ef-693437a82c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.822973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 788/1570] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.825922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9aa573-07b2-4999-9e81-9ddc7a4ce0a8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.837946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9aa573-07b2-4999-9e81-9ddc7a4ce0a8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.839037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5b6ea9fc-f892-45bb-93ef-693437a82c48 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:32.840462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5b6ea9fc-f892-45bb-93ef-693437a82c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:32.842892 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79af19f-96d4-43f7-a39d-1a520f6c38b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.843509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79af19f-96d4-43f7-a39d-1a520f6c38b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.843800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79af19f-96d4-43f7-a39d-1a520f6c38b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.844116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79af19f-96d4-43f7-a39d-1a520f6c38b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.844874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79af19f-96d4-43f7-a39d-1a520f6c38b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.846990 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79af19f-96d4-43f7-a39d-1a520f6c38b8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.849266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 783/1571] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:32.850034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ea9fc-f892-45bb-93ef-693437a82c48 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.860700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ea9fc-f892-45bb-93ef-693437a82c48 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.862043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 789/1572] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:32.862966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79af19f-96d4-43f7-a39d-1a520f6c38b8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.866558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4370939-9534-4647-943b-81f8d1611c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.867337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4370939-9534-4647-943b-81f8d1611c80 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.867623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4370939-9534-4647-943b-81f8d1611c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.867906 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4370939-9534-4647-943b-81f8d1611c80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.868655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4370939-9534-4647-943b-81f8d1611c80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:32.868967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4370939-9534-4647-943b-81f8d1611c80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.870576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4370939-9534-4647-943b-81f8d1611c80 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.873354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79af19f-96d4-43f7-a39d-1a520f6c38b8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.878001 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733393bb-0e72-41a0-9e8c-52bb90044ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.878609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733393bb-0e72-41a0-9e8c-52bb90044ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.878897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733393bb-0e72-41a0-9e8c-52bb90044ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.879175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733393bb-0e72-41a0-9e8c-52bb90044ed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.879865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733393bb-0e72-41a0-9e8c-52bb90044ed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:32.880203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733393bb-0e72-41a0-9e8c-52bb90044ed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.881781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733393bb-0e72-41a0-9e8c-52bb90044ed9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.889856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 784/1573] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:41:32.889856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4370939-9534-4647-943b-81f8d1611c80 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.894294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4370939-9534-4647-943b-81f8d1611c80 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.902070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 790/1574] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:41:32.902070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733393bb-0e72-41a0-9e8c-52bb90044ed9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.905482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52427b54-df03-4e5f-97be-2e6d960be9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.905633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733393bb-0e72-41a0-9e8c-52bb90044ed9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.906098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52427b54-df03-4e5f-97be-2e6d960be9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.906401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52427b54-df03-4e5f-97be-2e6d960be9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.906659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52427b54-df03-4e5f-97be-2e6d960be9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.907349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52427b54-df03-4e5f-97be-2e6d960be9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.909494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52427b54-df03-4e5f-97be-2e6d960be9e0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.910874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a2867-9cd0-4d2a-8d05-f7242ba6b0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.911467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a2867-9cd0-4d2a-8d05-f7242ba6b0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.911750 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a2867-9cd0-4d2a-8d05-f7242ba6b0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.912057 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a2867-9cd0-4d2a-8d05-f7242ba6b0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.912719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a2867-9cd0-4d2a-8d05-f7242ba6b0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.915045 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a2867-9cd0-4d2a-8d05-f7242ba6b0e8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:32.920400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 785/1575] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:32.920530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:32.921318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52427b54-df03-4e5f-97be-2e6d960be9e0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:32.931828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52427b54-df03-4e5f-97be-2e6d960be9e0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:32.936526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf17474-c649-4c16-847c-ea6010d2233b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:32.937123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf17474-c649-4c16-847c-ea6010d2233b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:32.937412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf17474-c649-4c16-847c-ea6010d2233b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:32.937691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf17474-c649-4c16-847c-ea6010d2233b admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:32.938374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf17474-c649-4c16-847c-ea6010d2233b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:32.940606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf17474-c649-4c16-847c-ea6010d2233b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.020176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 791/1576] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/projects => generated 393 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:33.021253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a2867-9cd0-4d2a-8d05-f7242ba6b0e8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.031940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 786/1577] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:32 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:33.033984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf17474-c649-4c16-847c-ea6010d2233b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.053102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:33.088085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a2867-9cd0-4d2a-8d05-f7242ba6b0e8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.091958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf17474-c649-4c16-847c-ea6010d2233b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.097217 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad827d1-b418-46ec-a427-f0184d5f9451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.097888 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad827d1-b418-46ec-a427-f0184d5f9451 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.098171 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad827d1-b418-46ec-a427-f0184d5f9451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.098445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad827d1-b418-46ec-a427-f0184d5f9451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.147603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ec3ae-4efb-4887-a3a0-aede463e4cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.148256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ec3ae-4efb-4887-a3a0-aede463e4cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.148564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ec3ae-4efb-4887-a3a0-aede463e4cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.148843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ec3ae-4efb-4887-a3a0-aede463e4cbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.149541 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ec3ae-4efb-4887-a3a0-aede463e4cbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:33.149850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ec3ae-4efb-4887-a3a0-aede463e4cbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.151475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ec3ae-4efb-4887-a3a0-aede463e4cbe admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.161985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad827d1-b418-46ec-a427-f0184d5f9451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.163465 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad827d1-b418-46ec-a427-f0184d5f9451 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.163649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 792/1578] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:41:33.164463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ec3ae-4efb-4887-a3a0-aede463e4cbe admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.173817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ec3ae-4efb-4887-a3a0-aede463e4cbe admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.177281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 787/1579] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:33.178060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad827d1-b418-46ec-a427-f0184d5f9451 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.178798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eca16-fab3-4c28-9b1f-59067696b99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.179514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eca16-fab3-4c28-9b1f-59067696b99f admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.179807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eca16-fab3-4c28-9b1f-59067696b99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.180116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eca16-fab3-4c28-9b1f-59067696b99f admin admin] PATH_INFO: `/v3/projects/4f9806b82f9e4854888cd294c7211cd1/users/811be94519f040e9abdc918dc3020567/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.187080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad827d1-b418-46ec-a427-f0184d5f9451 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.191892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94ff3d-e2c4-465d-811a-0144b2156ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.192532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94ff3d-e2c4-465d-811a-0144b2156ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.192810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94ff3d-e2c4-465d-811a-0144b2156ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.193079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94ff3d-e2c4-465d-811a-0144b2156ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.193736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94ff3d-e2c4-465d-811a-0144b2156ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.195642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94ff3d-e2c4-465d-811a-0144b2156ee2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.197079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eca16-fab3-4c28-9b1f-59067696b99f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9806b82f9e4854888cd294c7211cd1, user_id=811be94519f040e9abdc918dc3020567, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.199870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eca16-fab3-4c28-9b1f-59067696b99f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.275735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 793/1580] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:33 2022] PUT /identity/v3/projects/4f9806b82f9e4854888cd294c7211cd1/users/811be94519f040e9abdc918dc3020567/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:33.276323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 788/1581] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:33 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:33.277506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eca16-fab3-4c28-9b1f-59067696b99f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.280316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94ff3d-e2c4-465d-811a-0144b2156ee2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.348307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eca16-fab3-4c28-9b1f-59067696b99f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.353184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc49bf-579f-4a3d-a128-4f66f3771fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.353806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc49bf-579f-4a3d-a128-4f66f3771fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.354084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc49bf-579f-4a3d-a128-4f66f3771fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.354366 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc49bf-579f-4a3d-a128-4f66f3771fa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.355050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc49bf-579f-4a3d-a128-4f66f3771fa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.356985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94ff3d-e2c4-465d-811a-0144b2156ee2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.357117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc49bf-579f-4a3d-a128-4f66f3771fa1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.361556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de35d7f-a2ba-453f-8a18-4189b60bdecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.362991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de35d7f-a2ba-453f-8a18-4189b60bdecc admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.362991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de35d7f-a2ba-453f-8a18-4189b60bdecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.362991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de35d7f-a2ba-453f-8a18-4189b60bdecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.363393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de35d7f-a2ba-453f-8a18-4189b60bdecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.364997 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de35d7f-a2ba-453f-8a18-4189b60bdecc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.367304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:33.375077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 789/1582] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:33.375912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de35d7f-a2ba-453f-8a18-4189b60bdecc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.386104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de35d7f-a2ba-453f-8a18-4189b60bdecc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.391730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388293df-64ea-4bde-8320-d1aca603f192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.391730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388293df-64ea-4bde-8320-d1aca603f192 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.391730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388293df-64ea-4bde-8320-d1aca603f192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.392073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388293df-64ea-4bde-8320-d1aca603f192 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.393150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388293df-64ea-4bde-8320-d1aca603f192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.395045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388293df-64ea-4bde-8320-d1aca603f192 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.479141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 794/1583] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:33 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:33.479141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc49bf-579f-4a3d-a128-4f66f3771fa1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.493809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 790/1584] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:33 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:33.494554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388293df-64ea-4bde-8320-d1aca603f192 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.505466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388293df-64ea-4bde-8320-d1aca603f192 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.526955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:33.562900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc49bf-579f-4a3d-a128-4f66f3771fa1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.565199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cdf1b-ae80-459c-bfac-0b1f74fe0ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.565903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cdf1b-ae80-459c-bfac-0b1f74fe0ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.566192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cdf1b-ae80-459c-bfac-0b1f74fe0ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.566479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cdf1b-ae80-459c-bfac-0b1f74fe0ac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.567207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cdf1b-ae80-459c-bfac-0b1f74fe0ac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.568746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e83f8a-26f0-4792-9331-8cdffb90f2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.569192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cdf1b-ae80-459c-bfac-0b1f74fe0ac3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.569495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e83f8a-26f0-4792-9331-8cdffb90f2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.569793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e83f8a-26f0-4792-9331-8cdffb90f2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.570096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e83f8a-26f0-4792-9331-8cdffb90f2a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.570861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e83f8a-26f0-4792-9331-8cdffb90f2a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.572873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e83f8a-26f0-4792-9331-8cdffb90f2a6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.758005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 791/1585] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:33 2022] POST /identity/v3/projects => generated 385 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:33.761596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cdf1b-ae80-459c-bfac-0b1f74fe0ac3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.761900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 795/1586] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:33 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:33.763516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca22ed4-9599-4065-a599-5fec7f73e26d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.763816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca22ed4-9599-4065-a599-5fec7f73e26d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.764146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca22ed4-9599-4065-a599-5fec7f73e26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.790781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7ca22ed4-9599-4065-a599-5fec7f73e26d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:33.792011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7ca22ed4-9599-4065-a599-5fec7f73e26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:33.839119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cdf1b-ae80-459c-bfac-0b1f74fe0ac3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.843786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2e5db-d536-41da-95a9-48c40f27bbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.844447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2e5db-d536-41da-95a9-48c40f27bbd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.844724 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2e5db-d536-41da-95a9-48c40f27bbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.845001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2e5db-d536-41da-95a9-48c40f27bbd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.845666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2e5db-d536-41da-95a9-48c40f27bbd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.847735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2e5db-d536-41da-95a9-48c40f27bbd5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.853168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 796/1587] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:33.854052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca22ed4-9599-4065-a599-5fec7f73e26d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.866606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca22ed4-9599-4065-a599-5fec7f73e26d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.871272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889ead2-e53a-49a0-9122-5674a142972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.871962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889ead2-e53a-49a0-9122-5674a142972c admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.872280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889ead2-e53a-49a0-9122-5674a142972c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.872569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889ead2-e53a-49a0-9122-5674a142972c admin admin] PATH_INFO: `/v3/projects/3816f649c4c64f0ea9c0c5be8780794c/users/869b58008dd2449fbf31bafb8327fcb0/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.887733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889ead2-e53a-49a0-9122-5674a142972c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3816f649c4c64f0ea9c0c5be8780794c, user_id=869b58008dd2449fbf31bafb8327fcb0, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.890474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889ead2-e53a-49a0-9122-5674a142972c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:33.948385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 792/1588] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:33 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:33.949354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2e5db-d536-41da-95a9-48c40f27bbd5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:33.961009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2e5db-d536-41da-95a9-48c40f27bbd5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:33.965407 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a69d49-b50a-44c8-8b16-0a47eb06f8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:33.966180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69d49-b50a-44c8-8b16-0a47eb06f8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:33.966553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69d49-b50a-44c8-8b16-0a47eb06f8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:33.967952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69d49-b50a-44c8-8b16-0a47eb06f8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:33.967952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a69d49-b50a-44c8-8b16-0a47eb06f8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:33.969795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a69d49-b50a-44c8-8b16-0a47eb06f8a0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.007022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 797/1589] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:33 2022] PUT /identity/v3/projects/3816f649c4c64f0ea9c0c5be8780794c/users/869b58008dd2449fbf31bafb8327fcb0/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:34.008241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889ead2-e53a-49a0-9122-5674a142972c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.062305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 793/1590] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:33 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:34.063445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a69d49-b50a-44c8-8b16-0a47eb06f8a0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.073751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:34.073922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a69d49-b50a-44c8-8b16-0a47eb06f8a0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.078658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965de19-e85b-4730-8527-52a1ba9c9323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.078833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889ead2-e53a-49a0-9122-5674a142972c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.079393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965de19-e85b-4730-8527-52a1ba9c9323 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.079753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965de19-e85b-4730-8527-52a1ba9c9323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.080056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965de19-e85b-4730-8527-52a1ba9c9323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.080756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965de19-e85b-4730-8527-52a1ba9c9323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.082344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965de19-e85b-4730-8527-52a1ba9c9323 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.083505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df16273-fe0f-4f7f-9fc9-83964ab168f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.084401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df16273-fe0f-4f7f-9fc9-83964ab168f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.084688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df16273-fe0f-4f7f-9fc9-83964ab168f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.084954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df16273-fe0f-4f7f-9fc9-83964ab168f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.085642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df16273-fe0f-4f7f-9fc9-83964ab168f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.087217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df16273-fe0f-4f7f-9fc9-83964ab168f1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.093466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 794/1591] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:34.094246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965de19-e85b-4730-8527-52a1ba9c9323 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.100540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 798/1592] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:34.100540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df16273-fe0f-4f7f-9fc9-83964ab168f1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.100540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2df16273-fe0f-4f7f-9fc9-83964ab168f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:34.105558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965de19-e85b-4730-8527-52a1ba9c9323 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.111643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8ee75-1df7-47b7-a6e8-294e55336b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.112285 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8ee75-1df7-47b7-a6e8-294e55336b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.112569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8ee75-1df7-47b7-a6e8-294e55336b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.112844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8ee75-1df7-47b7-a6e8-294e55336b26 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.113538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8ee75-1df7-47b7-a6e8-294e55336b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.115485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8ee75-1df7-47b7-a6e8-294e55336b26 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.156061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df16273-fe0f-4f7f-9fc9-83964ab168f1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.160659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f39ad0-088d-4d0f-8a0e-48fdbeb34506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.161287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f39ad0-088d-4d0f-8a0e-48fdbeb34506 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.161572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f39ad0-088d-4d0f-8a0e-48fdbeb34506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.162240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f39ad0-088d-4d0f-8a0e-48fdbeb34506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.171622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f39ad0-088d-4d0f-8a0e-48fdbeb34506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.174588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f39ad0-088d-4d0f-8a0e-48fdbeb34506 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.183358 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 799/1593] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:34.184194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f39ad0-088d-4d0f-8a0e-48fdbeb34506 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.193057 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f39ad0-088d-4d0f-8a0e-48fdbeb34506 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.197664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4155c99-6d1d-42d1-92b7-bed5a904b2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.202977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4155c99-6d1d-42d1-92b7-bed5a904b2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.202977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4155c99-6d1d-42d1-92b7-bed5a904b2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.202977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4155c99-6d1d-42d1-92b7-bed5a904b2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.202977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4155c99-6d1d-42d1-92b7-bed5a904b2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.202977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4155c99-6d1d-42d1-92b7-bed5a904b2f2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.212161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 800/1594] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:34.213491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1db1f3-b13b-43c4-980c-ce8667dba33b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.213845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1db1f3-b13b-43c4-980c-ce8667dba33b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.214204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1db1f3-b13b-43c4-980c-ce8667dba33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.239212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 795/1595] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:34 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:34.239212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8ee75-1df7-47b7-a6e8-294e55336b26 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.249085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0e1db1f3-b13b-43c4-980c-ce8667dba33b None None] MFA Rules not processed for user `869b58008dd2449fbf31bafb8327fcb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:34.250320 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0e1db1f3-b13b-43c4-980c-ce8667dba33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:34.251069 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8ee75-1df7-47b7-a6e8-294e55336b26 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.256946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3712-c5e5-44bc-aa51-faf4cc3d9d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.258028 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3712-c5e5-44bc-aa51-faf4cc3d9d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.258028 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3712-c5e5-44bc-aa51-faf4cc3d9d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.258428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3712-c5e5-44bc-aa51-faf4cc3d9d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.259270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f3712-c5e5-44bc-aa51-faf4cc3d9d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.261217 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f3712-c5e5-44bc-aa51-faf4cc3d9d3b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.271940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 796/1596] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:34.272782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3712-c5e5-44bc-aa51-faf4cc3d9d3b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.282497 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3712-c5e5-44bc-aa51-faf4cc3d9d3b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.287846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2738a4-dff1-4339-bd7a-733ce2d96fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.288546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2738a4-dff1-4339-bd7a-733ce2d96fee admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.288896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2738a4-dff1-4339-bd7a-733ce2d96fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.289247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2738a4-dff1-4339-bd7a-733ce2d96fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.289934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2738a4-dff1-4339-bd7a-733ce2d96fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.291611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2738a4-dff1-4339-bd7a-733ce2d96fee admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.302789 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 797/1597] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:34.303673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2738a4-dff1-4339-bd7a-733ce2d96fee admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.313487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2738a4-dff1-4339-bd7a-733ce2d96fee admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.318441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e8a1e-c8f7-4f08-a7e7-4b67d264b580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.322909 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 801/1598] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:34.323048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e8a1e-c8f7-4f08-a7e7-4b67d264b580 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.323048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e8a1e-c8f7-4f08-a7e7-4b67d264b580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.323247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1db1f3-b13b-43c4-980c-ce8667dba33b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.323345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e8a1e-c8f7-4f08-a7e7-4b67d264b580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.323345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e8a1e-c8f7-4f08-a7e7-4b67d264b580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.323345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e8a1e-c8f7-4f08-a7e7-4b67d264b580 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.335459 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 798/1599] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:34.335569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1db1f3-b13b-43c4-980c-ce8667dba33b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.335661 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e8a1e-c8f7-4f08-a7e7-4b67d264b580 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.342223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7720554-908d-4b14-a5cc-c37339e7461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.342948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720554-908d-4b14-a5cc-c37339e7461b admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.343231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720554-908d-4b14-a5cc-c37339e7461b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.343511 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720554-908d-4b14-a5cc-c37339e7461b admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.344308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7720554-908d-4b14-a5cc-c37339e7461b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.346496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e8a1e-c8f7-4f08-a7e7-4b67d264b580 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.346695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7720554-908d-4b14-a5cc-c37339e7461b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.351175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621a6a6-2135-491d-8f7b-8c076ff808ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.352205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621a6a6-2135-491d-8f7b-8c076ff808ad admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.352205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621a6a6-2135-491d-8f7b-8c076ff808ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.352435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621a6a6-2135-491d-8f7b-8c076ff808ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.353080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621a6a6-2135-491d-8f7b-8c076ff808ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.354678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621a6a6-2135-491d-8f7b-8c076ff808ad admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.362236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 802/1600] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:34.362236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7720554-908d-4b14-a5cc-c37339e7461b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.367820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 799/1601] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:34.367820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621a6a6-2135-491d-8f7b-8c076ff808ad admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.374809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7720554-908d-4b14-a5cc-c37339e7461b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.378539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621a6a6-2135-491d-8f7b-8c076ff808ad admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.380184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efa2e4-1d5f-4619-b470-61b42f812977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.380956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa2e4-1d5f-4619-b470-61b42f812977 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.381230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa2e4-1d5f-4619-b470-61b42f812977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.381510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa2e4-1d5f-4619-b470-61b42f812977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.382211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efa2e4-1d5f-4619-b470-61b42f812977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.383840 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efa2e4-1d5f-4619-b470-61b42f812977 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.384703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703e69c-74d1-45ec-80da-023e2e4c59a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.385688 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e69c-74d1-45ec-80da-023e2e4c59a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.386660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e69c-74d1-45ec-80da-023e2e4c59a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.386920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e69c-74d1-45ec-80da-023e2e4c59a6 admin admin] PATH_INFO: `/v3/projects/cd88c4ac54be46ac9840241f6f4a7435/users/4db71a33784e47ba8a649936b31ec593/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.392142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:34.394095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 803/1602] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:34.394899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efa2e4-1d5f-4619-b470-61b42f812977 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.409727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703e69c-74d1-45ec-80da-023e2e4c59a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd88c4ac54be46ac9840241f6f4a7435, user_id=4db71a33784e47ba8a649936b31ec593, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.412535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703e69c-74d1-45ec-80da-023e2e4c59a6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.456202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efa2e4-1d5f-4619-b470-61b42f812977 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.458269 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6ab7b-1a9e-47f3-b35a-0afdde958cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.459257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6ab7b-1a9e-47f3-b35a-0afdde958cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.459608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6ab7b-1a9e-47f3-b35a-0afdde958cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.459955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6ab7b-1a9e-47f3-b35a-0afdde958cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.466006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:34.469730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6ab7b-1a9e-47f3-b35a-0afdde958cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.472160 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6ab7b-1a9e-47f3-b35a-0afdde958cd9 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.481339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 804/1603] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:34.482105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6ab7b-1a9e-47f3-b35a-0afdde958cd9 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.492167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6ab7b-1a9e-47f3-b35a-0afdde958cd9 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.496764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e294d-a71c-4dcd-849c-6af8992c692d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.497434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e294d-a71c-4dcd-849c-6af8992c692d admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.497719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e294d-a71c-4dcd-849c-6af8992c692d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.498013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e294d-a71c-4dcd-849c-6af8992c692d admin admin] PATH_INFO: `/v3/projects/f035e76db134496bbc7184d34d7f26d4/users/88d78d9c0369431d88ad85c960f6a0ba/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.517790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e294d-a71c-4dcd-849c-6af8992c692d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f035e76db134496bbc7184d34d7f26d4, user_id=88d78d9c0369431d88ad85c960f6a0ba, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.521220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e294d-a71c-4dcd-849c-6af8992c692d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.556091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 800/1604] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:34 2022] PUT /identity/v3/projects/cd88c4ac54be46ac9840241f6f4a7435/users/4db71a33784e47ba8a649936b31ec593/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:34.557304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703e69c-74d1-45ec-80da-023e2e4c59a6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.620513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 805/1605] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:34 2022] PUT /identity/v3/projects/f035e76db134496bbc7184d34d7f26d4/users/88d78d9c0369431d88ad85c960f6a0ba/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:34.621152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e294d-a71c-4dcd-849c-6af8992c692d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.647532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703e69c-74d1-45ec-80da-023e2e4c59a6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.725024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d5042-caba-40b8-9329-cf4e43210567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.726012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d5042-caba-40b8-9329-cf4e43210567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.726509 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d5042-caba-40b8-9329-cf4e43210567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.726953 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d5042-caba-40b8-9329-cf4e43210567 admin admin] PATH_INFO: `/v3/projects/d9a525f6d21a4f708cfc945db5e57f12/users/f7a5713174a74031b0d92610df1e28ef/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.749450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e294d-a71c-4dcd-849c-6af8992c692d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.755262 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d5042-caba-40b8-9329-cf4e43210567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a525f6d21a4f708cfc945db5e57f12, user_id=f7a5713174a74031b0d92610df1e28ef, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.758006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d5042-caba-40b8-9329-cf4e43210567 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.759778 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6500e4-a595-4e3d-9cd2-0564c839e92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.760490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6500e4-a595-4e3d-9cd2-0564c839e92e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.760771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6500e4-a595-4e3d-9cd2-0564c839e92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.761109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6500e4-a595-4e3d-9cd2-0564c839e92e admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.761801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6500e4-a595-4e3d-9cd2-0564c839e92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.764097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6500e4-a595-4e3d-9cd2-0564c839e92e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.789115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 806/1606] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:34.789956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6500e4-a595-4e3d-9cd2-0564c839e92e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:34.804881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6500e4-a595-4e3d-9cd2-0564c839e92e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:34.810812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb2f1-f2e2-459b-bf6e-0e944babb6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:34.811636 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb2f1-f2e2-459b-bf6e-0e944babb6ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.812015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb2f1-f2e2-459b-bf6e-0e944babb6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.812418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb2f1-f2e2-459b-bf6e-0e944babb6ce admin admin] PATH_INFO: `/v3/projects/5dc837002aaa47979fe783bc0bc1cbf2/users/92fa59772a574f42b96a0776c269b93c/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.835958 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eb2f1-f2e2-459b-bf6e-0e944babb6ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc837002aaa47979fe783bc0bc1cbf2, user_id=92fa59772a574f42b96a0776c269b93c, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:34.838811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eb2f1-f2e2-459b-bf6e-0e944babb6ce admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:34.853804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 801/1607] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:34 2022] PUT /identity/v3/projects/d9a525f6d21a4f708cfc945db5e57f12/users/f7a5713174a74031b0d92610df1e28ef/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:34.855894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e46c7-779f-4924-ae31-1b1279403e58 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.856741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e46c7-779f-4924-ae31-1b1279403e58 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.856741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e46c7-779f-4924-ae31-1b1279403e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.886716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9d5e46c7-779f-4924-ae31-1b1279403e58 None None] MFA Rules not processed for user `4db71a33784e47ba8a649936b31ec593`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:34.888047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9d5e46c7-779f-4924-ae31-1b1279403e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:34.943313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 807/1608] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:34 2022] PUT /identity/v3/projects/5dc837002aaa47979fe783bc0bc1cbf2/users/92fa59772a574f42b96a0776c269b93c/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:34.944794 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b7dce-989b-4a35-8db9-7ef35fb16715 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:34.945196 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b7dce-989b-4a35-8db9-7ef35fb16715 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:34.945565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b7dce-989b-4a35-8db9-7ef35fb16715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:34.992975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-858b7dce-989b-4a35-8db9-7ef35fb16715 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:34.994450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-858b7dce-989b-4a35-8db9-7ef35fb16715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:34.998606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 802/1609] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:34.999773 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e46c7-779f-4924-ae31-1b1279403e58 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:35.005024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:35.100939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e46c7-779f-4924-ae31-1b1279403e58 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:35.162287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 808/1610] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:35.163835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c83b-2988-4ff7-8789-ddb13251f135 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:35.164466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c83b-2988-4ff7-8789-ddb13251f135 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:35.164748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c83b-2988-4ff7-8789-ddb13251f135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:35.168411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024453f-7ce3-4c10-8891-507684289bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:35.169336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024453f-7ce3-4c10-8891-507684289bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:35.169782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024453f-7ce3-4c10-8891-507684289bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:35.170280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024453f-7ce3-4c10-8891-507684289bd1 admin admin] PATH_INFO: `/v3/projects/fda70ea579f941e6aefa8f06367ab3c9/users/5509519254574e50a70c84ae895acd3f/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:35.200594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fa78c83b-2988-4ff7-8789-ddb13251f135 None None] MFA Rules not processed for user `f7a5713174a74031b0d92610df1e28ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:35.202819 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fa78c83b-2988-4ff7-8789-ddb13251f135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:35.214322 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024453f-7ce3-4c10-8891-507684289bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fda70ea579f941e6aefa8f06367ab3c9, user_id=5509519254574e50a70c84ae895acd3f, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:35.219303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024453f-7ce3-4c10-8891-507684289bd1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:35.272507 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:35.309006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 803/1611] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:34 2022] PUT /identity/v3/projects/fda70ea579f941e6aefa8f06367ab3c9/users/5509519254574e50a70c84ae895acd3f/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:35.310284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41474c2-c961-4c4e-b5f0-9153c23aa56f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:35.310570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41474c2-c961-4c4e-b5f0-9153c23aa56f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:35.310843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41474c2-c961-4c4e-b5f0-9153c23aa56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:35.356357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 809/1612] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:35.357660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78c83b-2988-4ff7-8789-ddb13251f135 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:35.362726 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a41474c2-c961-4c4e-b5f0-9153c23aa56f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:35.364082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a41474c2-c961-4c4e-b5f0-9153c23aa56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:35.370056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78c83b-2988-4ff7-8789-ddb13251f135 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:35.438484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 804/1613] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:35.440092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41474c2-c961-4c4e-b5f0-9153c23aa56f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:35.461806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86522cdb-3734-43dc-86cd-866228050aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:35.462894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86522cdb-3734-43dc-86cd-866228050aab admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:35.463358 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86522cdb-3734-43dc-86cd-866228050aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:35.463811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86522cdb-3734-43dc-86cd-866228050aab admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:35.465394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86522cdb-3734-43dc-86cd-866228050aab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:35.469626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86522cdb-3734-43dc-86cd-866228050aab admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:35.487595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:35.533560 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41474c2-c961-4c4e-b5f0-9153c23aa56f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:35.553455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0012f4-bf65-4d9e-a04f-09901cf6c78a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:35.554569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0012f4-bf65-4d9e-a04f-09901cf6c78a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:35.558277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0012f4-bf65-4d9e-a04f-09901cf6c78a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:35.558277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0012f4-bf65-4d9e-a04f-09901cf6c78a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:35.570140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0012f4-bf65-4d9e-a04f-09901cf6c78a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:35.572476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0012f4-bf65-4d9e-a04f-09901cf6c78a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:35.589820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 810/1614] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:35 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:35.591429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51423a7a-d24c-4822-af7e-2d4e0acfda1b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:35.592750 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51423a7a-d24c-4822-af7e-2d4e0acfda1b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:35.593763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51423a7a-d24c-4822-af7e-2d4e0acfda1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:35.612693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 805/1615] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:35.613558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0012f4-bf65-4d9e-a04f-09901cf6c78a service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:35.638217 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0012f4-bf65-4d9e-a04f-09901cf6c78a service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:35.647972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-51423a7a-d24c-4822-af7e-2d4e0acfda1b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:35.649265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51423a7a-d24c-4822-af7e-2d4e0acfda1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:35.741823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda326e6-9215-466b-8724-95629339e556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:35.766223 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda326e6-9215-466b-8724-95629339e556 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:35.767185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda326e6-9215-466b-8724-95629339e556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:35.770511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda326e6-9215-466b-8724-95629339e556 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:35.771503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda326e6-9215-466b-8724-95629339e556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:35.775599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda326e6-9215-466b-8724-95629339e556 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:35.777739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 811/1616] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:35.778885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51423a7a-d24c-4822-af7e-2d4e0acfda1b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:35.990878 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 806/1617] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:35 2022] POST /identity/v3/projects => generated 379 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:35.990878 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda326e6-9215-466b-8724-95629339e556 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.001046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51423a7a-d24c-4822-af7e-2d4e0acfda1b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:36.001046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:36.015927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4e925-b07c-4aaf-8040-c5c5643cd213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:36.016631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4e925-b07c-4aaf-8040-c5c5643cd213 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.016915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4e925-b07c-4aaf-8040-c5c5643cd213 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.017821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4e925-b07c-4aaf-8040-c5c5643cd213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.051737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda326e6-9215-466b-8724-95629339e556 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:36.059416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674c92d-1b2b-4dce-850e-c458df12b0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:36.060056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c92d-1b2b-4dce-850e-c458df12b0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.061431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c92d-1b2b-4dce-850e-c458df12b0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.061721 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c92d-1b2b-4dce-850e-c458df12b0b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.064979 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674c92d-1b2b-4dce-850e-c458df12b0b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:36.070544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674c92d-1b2b-4dce-850e-c458df12b0b4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:36.112381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4e925-b07c-4aaf-8040-c5c5643cd213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:36.113872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4e925-b07c-4aaf-8040-c5c5643cd213 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:36.141847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 812/1618] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:36.141847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4e925-b07c-4aaf-8040-c5c5643cd213 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.148202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4e925-b07c-4aaf-8040-c5c5643cd213 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:36.152238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d4776-fa34-430d-bb1b-a902ad5df436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:36.152872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d4776-fa34-430d-bb1b-a902ad5df436 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.153210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d4776-fa34-430d-bb1b-a902ad5df436 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.153495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d4776-fa34-430d-bb1b-a902ad5df436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.167889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 807/1619] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:35 2022] POST /identity/v3/users => generated 299 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:36.168701 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674c92d-1b2b-4dce-850e-c458df12b0b4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.174945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d4776-fa34-430d-bb1b-a902ad5df436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:36.176536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d4776-fa34-430d-bb1b-a902ad5df436 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:36.180216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674c92d-1b2b-4dce-850e-c458df12b0b4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:36.187011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f072b-da65-4464-b571-546882a52a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:36.187570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 813/1620] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:36.188736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743ba07-8636-435c-b24f-aeefcc0a3c27 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.189175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743ba07-8636-435c-b24f-aeefcc0a3c27 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.189651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743ba07-8636-435c-b24f-aeefcc0a3c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.193933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f072b-da65-4464-b571-546882a52a7f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.198117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f072b-da65-4464-b571-546882a52a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.198117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f072b-da65-4464-b571-546882a52a7f admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.198117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f072b-da65-4464-b571-546882a52a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:36.202878 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f072b-da65-4464-b571-546882a52a7f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:36.231694 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0743ba07-8636-435c-b24f-aeefcc0a3c27 None None] MFA Rules not processed for user `811be94519f040e9abdc918dc3020567`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:36.233267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0743ba07-8636-435c-b24f-aeefcc0a3c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:36.282016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 808/1621] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:36 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:36.283882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f072b-da65-4464-b571-546882a52a7f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.300683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f072b-da65-4464-b571-546882a52a7f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:36.306203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789936d8-437a-462f-b2c5-7035bc03310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:36.306420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789936d8-437a-462f-b2c5-7035bc03310a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.306573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789936d8-437a-462f-b2c5-7035bc03310a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.306984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789936d8-437a-462f-b2c5-7035bc03310a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.307680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789936d8-437a-462f-b2c5-7035bc03310a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:36.313606 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789936d8-437a-462f-b2c5-7035bc03310a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:36.318450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 814/1622] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:36 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:36.319499 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743ba07-8636-435c-b24f-aeefcc0a3c27 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.328587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 809/1623] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:36.334083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789936d8-437a-462f-b2c5-7035bc03310a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.335079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743ba07-8636-435c-b24f-aeefcc0a3c27 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:36.342124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441d191-a77a-48b8-8c6d-649286d7b4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:36.342124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d191-a77a-48b8-8c6d-649286d7b4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.342598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d191-a77a-48b8-8c6d-649286d7b4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.343032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d191-a77a-48b8-8c6d-649286d7b4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.343906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441d191-a77a-48b8-8c6d-649286d7b4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:36.345872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441d191-a77a-48b8-8c6d-649286d7b4c6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:36.349701 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789936d8-437a-462f-b2c5-7035bc03310a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:36.356444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b335df0-f3ff-4504-9ca1-43c9fe18b1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:36.360349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 815/1624] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:36.360511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b335df0-f3ff-4504-9ca1-43c9fe18b1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.360609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441d191-a77a-48b8-8c6d-649286d7b4c6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.360698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b335df0-f3ff-4504-9ca1-43c9fe18b1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.360698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b335df0-f3ff-4504-9ca1-43c9fe18b1f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.360698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b335df0-f3ff-4504-9ca1-43c9fe18b1f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:36.362561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b335df0-f3ff-4504-9ca1-43c9fe18b1f9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:36.375273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441d191-a77a-48b8-8c6d-649286d7b4c6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:36.383163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d930a-7943-46b7-aa7a-4a215576100a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:36.384158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d930a-7943-46b7-aa7a-4a215576100a admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.384733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d930a-7943-46b7-aa7a-4a215576100a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.385237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d930a-7943-46b7-aa7a-4a215576100a admin admin] PATH_INFO: `/v3/projects/090cba77061046a3bd569ed7da54e2fc/users/e99714658955413c8cb7134c04f39cf2/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.403359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d930a-7943-46b7-aa7a-4a215576100a admin admin] RBAC: Authorizing `identity:create_grant(project_id=090cba77061046a3bd569ed7da54e2fc, user_id=e99714658955413c8cb7134c04f39cf2, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:36.406326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d930a-7943-46b7-aa7a-4a215576100a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:36.567759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:36.577147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 816/1625] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:36 2022] PUT /identity/v3/projects/090cba77061046a3bd569ed7da54e2fc/users/e99714658955413c8cb7134c04f39cf2/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:36.578940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d930a-7943-46b7-aa7a-4a215576100a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.585739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 810/1626] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:36 2022] POST /identity/v3/projects => generated 403 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:36.585739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b335df0-f3ff-4504-9ca1-43c9fe18b1f9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.598295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:36.742569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b335df0-f3ff-4504-9ca1-43c9fe18b1f9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:36.744514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d930a-7943-46b7-aa7a-4a215576100a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:36.757598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27eb965-2aa1-4639-bf7e-d6f029a0771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:36.758247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27eb965-2aa1-4639-bf7e-d6f029a0771c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.758531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27eb965-2aa1-4639-bf7e-d6f029a0771c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.758805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27eb965-2aa1-4639-bf7e-d6f029a0771c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.759623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27eb965-2aa1-4639-bf7e-d6f029a0771c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:36.760470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f980f-9bb7-465c-a231-58c72e9068ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:36.761220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f980f-9bb7-465c-a231-58c72e9068ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.761335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27eb965-2aa1-4639-bf7e-d6f029a0771c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:36.761501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f980f-9bb7-465c-a231-58c72e9068ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.761789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f980f-9bb7-465c-a231-58c72e9068ba admin admin] PATH_INFO: `/v3/projects/f8f739e8d70a4a69971c92a665fce2cc/users/6404342fec824715b6cedd8926132163/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.794891 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f980f-9bb7-465c-a231-58c72e9068ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8f739e8d70a4a69971c92a665fce2cc, user_id=6404342fec824715b6cedd8926132163, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:36.796950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 811/1627] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:36.797951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27eb965-2aa1-4639-bf7e-d6f029a0771c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.798083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f980f-9bb7-465c-a231-58c72e9068ba admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:36.809626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27eb965-2aa1-4639-bf7e-d6f029a0771c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:36.817625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d2c75-9d2c-428a-82e3-d5f5f32d2205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:36.818739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2c75-9d2c-428a-82e3-d5f5f32d2205 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:36.819238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2c75-9d2c-428a-82e3-d5f5f32d2205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:36.819748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2c75-9d2c-428a-82e3-d5f5f32d2205 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:36.820688 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d2c75-9d2c-428a-82e3-d5f5f32d2205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:36.822990 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d2c75-9d2c-428a-82e3-d5f5f32d2205 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:36.933757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 817/1628] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:36 2022] PUT /identity/v3/projects/f8f739e8d70a4a69971c92a665fce2cc/users/6404342fec824715b6cedd8926132163/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:36.941582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f980f-9bb7-465c-a231-58c72e9068ba admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.941714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 812/1629] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:36 2022] POST /identity/v3/users => generated 308 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:36.941714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d2c75-9d2c-428a-82e3-d5f5f32d2205 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:36.962315 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d2c75-9d2c-428a-82e3-d5f5f32d2205 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:37.069622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7cbbc-f342-46e8-a80d-d285de0c7116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:37.069867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f980f-9bb7-465c-a231-58c72e9068ba admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:37.071200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7cbbc-f342-46e8-a80d-d285de0c7116 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:37.071854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7cbbc-f342-46e8-a80d-d285de0c7116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:37.072381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7cbbc-f342-46e8-a80d-d285de0c7116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:37.073355 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7cbbc-f342-46e8-a80d-d285de0c7116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:37.075621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7409c-f300-4799-99d1-7b2ce0969c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:37.076183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7cbbc-f342-46e8-a80d-d285de0c7116 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:37.076334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7409c-f300-4799-99d1-7b2ce0969c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:37.076608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7409c-f300-4799-99d1-7b2ce0969c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:37.076891 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7409c-f300-4799-99d1-7b2ce0969c26 admin admin] PATH_INFO: `/v3/projects/090cba77061046a3bd569ed7da54e2fc/users/e99714658955413c8cb7134c04f39cf2/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:37.078875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7409c-f300-4799-99d1-7b2ce0969c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=090cba77061046a3bd569ed7da54e2fc, user_id=e99714658955413c8cb7134c04f39cf2, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:37.081858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7409c-f300-4799-99d1-7b2ce0969c26 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:37.099237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 818/1630] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:36 2022] PUT /identity/v3/projects/090cba77061046a3bd569ed7da54e2fc/users/e99714658955413c8cb7134c04f39cf2/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:37.100008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 813/1631] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:37.100143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7409c-f300-4799-99d1-7b2ce0969c26 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:37.103858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7cbbc-f342-46e8-a80d-d285de0c7116 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:37.127022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7cbbc-f342-46e8-a80d-d285de0c7116 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:37.211127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7409c-f300-4799-99d1-7b2ce0969c26 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:37.219737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ae914-f294-4f41-a5f0-c5e11445ac56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:37.219965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe98d6-722d-4604-bffa-35e8e597d79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:37.221160 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe98d6-722d-4604-bffa-35e8e597d79a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:37.221493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe98d6-722d-4604-bffa-35e8e597d79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:37.222051 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ae914-f294-4f41-a5f0-c5e11445ac56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:37.222279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe98d6-722d-4604-bffa-35e8e597d79a admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:37.222763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ae914-f294-4f41-a5f0-c5e11445ac56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:37.223010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe98d6-722d-4604-bffa-35e8e597d79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:37.224313 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ae914-f294-4f41-a5f0-c5e11445ac56 admin admin] PATH_INFO: `/v3/projects/f555268a8f9442fe94f41f8ad411ec04/users/1272d313c8f34f42bfa34e245471a2fb/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:37.226055 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe98d6-722d-4604-bffa-35e8e597d79a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:37.238718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 819/1632] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:37.240098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ad4f9-f346-4099-bfe5-4ffb9bad50d3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:37.240394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ad4f9-f346-4099-bfe5-4ffb9bad50d3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:37.240678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ad4f9-f346-4099-bfe5-4ffb9bad50d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:37.247655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ae914-f294-4f41-a5f0-c5e11445ac56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f555268a8f9442fe94f41f8ad411ec04, user_id=1272d313c8f34f42bfa34e245471a2fb, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:37.253376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ae914-f294-4f41-a5f0-c5e11445ac56 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:37.272349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-798ad4f9-f346-4099-bfe5-4ffb9bad50d3 None None] MFA Rules not processed for user `e99714658955413c8cb7134c04f39cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:37.273301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-798ad4f9-f346-4099-bfe5-4ffb9bad50d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:37.360553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 820/1633] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:37.360553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ad4f9-f346-4099-bfe5-4ffb9bad50d3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:37.380133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 814/1634] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:37 2022] PUT /identity/v3/projects/f555268a8f9442fe94f41f8ad411ec04/users/1272d313c8f34f42bfa34e245471a2fb/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:41:37.380133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ae914-f294-4f41-a5f0-c5e11445ac56 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:37.380552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ad4f9-f346-4099-bfe5-4ffb9bad50d3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:37.438330 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba3c71-653d-4087-be1d-370d95bd279f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:37.439285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba3c71-653d-4087-be1d-370d95bd279f admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:37.439850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba3c71-653d-4087-be1d-370d95bd279f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:37.440310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba3c71-653d-4087-be1d-370d95bd279f admin admin] PATH_INFO: `/v3/projects/f8f739e8d70a4a69971c92a665fce2cc/users/6404342fec824715b6cedd8926132163/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:37.442490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba3c71-653d-4087-be1d-370d95bd279f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8f739e8d70a4a69971c92a665fce2cc, user_id=6404342fec824715b6cedd8926132163, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:37.445836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba3c71-653d-4087-be1d-370d95bd279f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:37.467725 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 821/1635] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:37 2022] PUT /identity/v3/projects/f8f739e8d70a4a69971c92a665fce2cc/users/6404342fec824715b6cedd8926132163/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:37.467725 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba3c71-653d-4087-be1d-370d95bd279f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:37.506233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ae914-f294-4f41-a5f0-c5e11445ac56 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:37.589393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:37.592084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba3c71-653d-4087-be1d-370d95bd279f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:37.600243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17701a75-91d8-4fc1-b7d7-80cf6c4888f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:37.600243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17701a75-91d8-4fc1-b7d7-80cf6c4888f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:37.600243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17701a75-91d8-4fc1-b7d7-80cf6c4888f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:37.600667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9b16c-9f68-4050-9390-3e22224e1af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:37.600780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17701a75-91d8-4fc1-b7d7-80cf6c4888f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:37.600871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9b16c-9f68-4050-9390-3e22224e1af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:37.600871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9b16c-9f68-4050-9390-3e22224e1af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:37.600871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9b16c-9f68-4050-9390-3e22224e1af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:37.600871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9b16c-9f68-4050-9390-3e22224e1af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:37.602527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9b16c-9f68-4050-9390-3e22224e1af0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:37.617385 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 822/1636] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:37.623214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137526f-2194-4f73-8fc9-5cd20a9b19b1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:37.623214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137526f-2194-4f73-8fc9-5cd20a9b19b1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:37.623475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137526f-2194-4f73-8fc9-5cd20a9b19b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:37.652768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:37.704836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5137526f-2194-4f73-8fc9-5cd20a9b19b1 None None] MFA Rules not processed for user `6404342fec824715b6cedd8926132163`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:37.709581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5137526f-2194-4f73-8fc9-5cd20a9b19b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:37.734733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17701a75-91d8-4fc1-b7d7-80cf6c4888f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:37.738306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17701a75-91d8-4fc1-b7d7-80cf6c4888f1 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:37.753023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 815/1637] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:37.753930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17701a75-91d8-4fc1-b7d7-80cf6c4888f1 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:37.770272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17701a75-91d8-4fc1-b7d7-80cf6c4888f1 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:37.781739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074119e-0583-47ba-b4cb-de065f59afae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:37.782580 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074119e-0583-47ba-b4cb-de065f59afae admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:37.782981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074119e-0583-47ba-b4cb-de065f59afae admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:37.784195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074119e-0583-47ba-b4cb-de065f59afae admin admin] PATH_INFO: `/v3/projects/f555268a8f9442fe94f41f8ad411ec04/users/1272d313c8f34f42bfa34e245471a2fb/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:37.787747 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074119e-0583-47ba-b4cb-de065f59afae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f555268a8f9442fe94f41f8ad411ec04, user_id=1272d313c8f34f42bfa34e245471a2fb, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:37.791779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074119e-0583-47ba-b4cb-de065f59afae admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:37.813402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 816/1638] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:37 2022] PUT /identity/v3/projects/f555268a8f9442fe94f41f8ad411ec04/users/1272d313c8f34f42bfa34e245471a2fb/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:37.835357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 823/1639] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:37.915364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074119e-0583-47ba-b4cb-de065f59afae admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:38.058043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074119e-0583-47ba-b4cb-de065f59afae admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:38.064619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade61481-c7ad-4020-b097-36ef406fd1e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:38.066592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade61481-c7ad-4020-b097-36ef406fd1e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:38.067002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade61481-c7ad-4020-b097-36ef406fd1e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:38.067325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade61481-c7ad-4020-b097-36ef406fd1e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:38.199100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade61481-c7ad-4020-b097-36ef406fd1e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:38.200660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade61481-c7ad-4020-b097-36ef406fd1e7 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:38.222911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 817/1640] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:38.301338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f966a2-1579-4e6a-a7da-fa054e7f93bd None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:38.301981 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f966a2-1579-4e6a-a7da-fa054e7f93bd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:38.302451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f966a2-1579-4e6a-a7da-fa054e7f93bd None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:38.304746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 824/1641] 10.222.0.64 () {54 vars in 962 bytes} [Thu Aug 4 07:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:41:38.315875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d41f0-7284-4d6f-82e9-c153e880283c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:38.315875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d41f0-7284-4d6f-82e9-c153e880283c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:38.316195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d41f0-7284-4d6f-82e9-c153e880283c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:38.430146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0b6d41f0-7284-4d6f-82e9-c153e880283c None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:38.432353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0b6d41f0-7284-4d6f-82e9-c153e880283c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:38.457528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:38.520533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f966a2-1579-4e6a-a7da-fa054e7f93bd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:38.818118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f966a2-1579-4e6a-a7da-fa054e7f93bd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:38.818118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d26a2a-0f1a-4f5b-9662-7218e0441f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:38.818118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d26a2a-0f1a-4f5b-9662-7218e0441f48 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:38.818118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d26a2a-0f1a-4f5b-9662-7218e0441f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:38.825457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d26a2a-0f1a-4f5b-9662-7218e0441f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:38.827955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 818/1642] 10.222.0.64 () {58 vars in 1059 bytes} [Thu Aug 4 07:41:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:41:38.827955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0791fbc-88c0-4c30-8aeb-6f9aa73d6212 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:38.827955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0791fbc-88c0-4c30-8aeb-6f9aa73d6212 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:38.827955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0791fbc-88c0-4c30-8aeb-6f9aa73d6212 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:38.827955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 819/1643] 10.222.0.64 () {54 vars in 912 bytes} [Thu Aug 4 07:41:38 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:41:38.827955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97d7c-1dfb-43b5-b5e0-2f4621dcc545 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:38.827955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97d7c-1dfb-43b5-b5e0-2f4621dcc545 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:38.831900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97d7c-1dfb-43b5-b5e0-2f4621dcc545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:38.832500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:38.832901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1ea97d7c-1dfb-43b5-b5e0-2f4621dcc545 None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:38.832901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1ea97d7c-1dfb-43b5-b5e0-2f4621dcc545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:38.833256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d26a2a-0f1a-4f5b-9662-7218e0441f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:38.833256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d26a2a-0f1a-4f5b-9662-7218e0441f48 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:38.833589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 820/1644] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:41:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:38.833589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97d7c-1dfb-43b5-b5e0-2f4621dcc545 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:38.834177 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 825/1645] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:38.835267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97d7c-1dfb-43b5-b5e0-2f4621dcc545 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:38.835267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d1e49-1bbf-4dae-8ccf-404f07336ce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:38.835267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1e49-1bbf-4dae-8ccf-404f07336ce2 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:38.835267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1e49-1bbf-4dae-8ccf-404f07336ce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:38.837914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1e49-1bbf-4dae-8ccf-404f07336ce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:38.837914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d1e49-1bbf-4dae-8ccf-404f07336ce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:38.837914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d1e49-1bbf-4dae-8ccf-404f07336ce2 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:38.837914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 821/1646] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:40.408210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d26a2a-0f1a-4f5b-9662-7218e0441f48 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:40.556944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d26a2a-0f1a-4f5b-9662-7218e0441f48 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:40.566523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:40.567338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:40.567723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:40.568046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] PATH_INFO: `/v3/users/4db71a33784e47ba8a649936b31ec593` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:40.569394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db71a33784e47ba8a649936b31ec593)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:40.573342 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:40.820262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db71a33784e47ba8a649936b31ec593'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:40.821010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db71a33784e47ba8a649936b31ec593'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:40.825429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db71a33784e47ba8a649936b31ec593'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:40.844555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:40.853482 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db71a33784e47ba8a649936b31ec593'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:40.944114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db71a33784e47ba8a649936b31ec593'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:40.968605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 826/1647] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:41:40 2022] DELETE /identity/v3/users/4db71a33784e47ba8a649936b31ec593 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:40.971003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7a0a-67ff-4436-a6f9-bcc855bc0cc3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:40.971580 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7a0a-67ff-4436-a6f9-bcc855bc0cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:40.971580 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7a0a-67ff-4436-a6f9-bcc855bc0cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:41.004469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cf5d7a0a-67ff-4436-a6f9-bcc855bc0cc3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:41.005730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cf5d7a0a-67ff-4436-a6f9-bcc855bc0cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:41.142898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:41.159614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 822/1648] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:41.426486 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:41.427406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:41.599623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86440b0-6366-4390-8898-9b95729af405 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:41.612034 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddee68c-4014-40ce-a7fc-ef1c6119fffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:41.612690 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddee68c-4014-40ce-a7fc-ef1c6119fffd service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:41.612978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddee68c-4014-40ce-a7fc-ef1c6119fffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:41.613314 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddee68c-4014-40ce-a7fc-ef1c6119fffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:41.637146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddee68c-4014-40ce-a7fc-ef1c6119fffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:41.640967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddee68c-4014-40ce-a7fc-ef1c6119fffd service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:41.679368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 827/1649] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:41.991230 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cb72f-c58f-40e8-b922-7582b94f58bb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:42.006932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cb72f-c58f-40e8-b922-7582b94f58bb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:42.006932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cb72f-c58f-40e8-b922-7582b94f58bb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:42.008793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 823/1650] 10.222.0.64 () {54 vars in 962 bytes} [Thu Aug 4 07:41:41 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:41:42.025634 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212a06c-dd78-41ae-b85f-6a5202273cd8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:42.026241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212a06c-dd78-41ae-b85f-6a5202273cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:42.026835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212a06c-dd78-41ae-b85f-6a5202273cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:42.071394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0212a06c-dd78-41ae-b85f-6a5202273cd8 None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:42.076982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0212a06c-dd78-41ae-b85f-6a5202273cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:42.300079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 828/1651] 10.222.0.64 () {58 vars in 1059 bytes} [Thu Aug 4 07:41:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:42.306893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de230fe1-4c94-4cff-a01f-9fa42c466f79 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:42.308489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de230fe1-4c94-4cff-a01f-9fa42c466f79 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:42.308975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de230fe1-4c94-4cff-a01f-9fa42c466f79 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:42.310886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 824/1652] 10.222.0.64 () {54 vars in 912 bytes} [Thu Aug 4 07:41:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:41:42.321122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1ddfc-ec0b-43cf-8803-f9d994675332 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:42.321535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1ddfc-ec0b-43cf-8803-f9d994675332 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:42.321917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1ddfc-ec0b-43cf-8803-f9d994675332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:42.363434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2ed1ddfc-ec0b-43cf-8803-f9d994675332 None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:42.380447 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2ed1ddfc-ec0b-43cf-8803-f9d994675332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:42.397632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 829/1653] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:41:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:42.417960 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de230fe1-4c94-4cff-a01f-9fa42c466f79 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:42.420583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-de230fe1-4c94-4cff-a01f-9fa42c466f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:42.527678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de230fe1-4c94-4cff-a01f-9fa42c466f79 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:42.534743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ba2da-ece6-423a-aed8-d9e153e00d23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:42.535502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba2da-ece6-423a-aed8-d9e153e00d23 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:42.535849 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba2da-ece6-423a-aed8-d9e153e00d23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:42.537517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba2da-ece6-423a-aed8-d9e153e00d23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:42.556858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ba2da-ece6-423a-aed8-d9e153e00d23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:42.559027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ba2da-ece6-423a-aed8-d9e153e00d23 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:42.574539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 825/1654] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:42.886550 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf042d-f7d6-4b81-b39b-356579fd85c8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:42.887088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf042d-f7d6-4b81-b39b-356579fd85c8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:42.887088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf042d-f7d6-4b81-b39b-356579fd85c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:42.941270 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d1bf042d-f7d6-4b81-b39b-356579fd85c8 None None] MFA Rules not processed for user `1272d313c8f34f42bfa34e245471a2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:42.942946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d1bf042d-f7d6-4b81-b39b-356579fd85c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:43.062802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:43.106564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 830/1655] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:42 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:43.334249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ba2da-ece6-423a-aed8-d9e153e00d23 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:43.335916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-005ba2da-ece6-423a-aed8-d9e153e00d23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:43.393551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ba2da-ece6-423a-aed8-d9e153e00d23 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:43.417539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:43.422702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:43.422702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:43.422702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] PATH_INFO: `/v3/projects/cd88c4ac54be46ac9840241f6f4a7435` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:43.428995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd88c4ac54be46ac9840241f6f4a7435)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:43.430961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:43.640656 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd88c4ac54be46ac9840241f6f4a7435', 'request_id': 'req-fe193ab4-a64e-48b6-80c7-5d2612806a08'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:43.760857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd88c4ac54be46ac9840241f6f4a7435', 'request_id': 'req-fe193ab4-a64e-48b6-80c7-5d2612806a08'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:43.787427 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd88c4ac54be46ac9840241f6f4a7435', 'request_id': 'req-fe193ab4-a64e-48b6-80c7-5d2612806a08'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:43.825670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] The token cache is being invalidate because project cd88c4ac54be46ac9840241f6f4a7435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:41:43.828852 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:43.831223 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe193ab4-a64e-48b6-80c7-5d2612806a08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:43.843337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf042d-f7d6-4b81-b39b-356579fd85c8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:43.844221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 826/1656] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:41:43 2022] DELETE /identity/v3/projects/cd88c4ac54be46ac9840241f6f4a7435 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:43.868012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf042d-f7d6-4b81-b39b-356579fd85c8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:43.999801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777dacc-4e03-4ed8-9d9e-136ee013cf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:44.001575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777dacc-4e03-4ed8-9d9e-136ee013cf82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:44.005087 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777dacc-4e03-4ed8-9d9e-136ee013cf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:44.005532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777dacc-4e03-4ed8-9d9e-136ee013cf82 admin admin] PATH_INFO: `/v3/users/e99714658955413c8cb7134c04f39cf2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:44.008968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777dacc-4e03-4ed8-9d9e-136ee013cf82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e99714658955413c8cb7134c04f39cf2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:44.013612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777dacc-4e03-4ed8-9d9e-136ee013cf82 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:44.369143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3777dacc-4e03-4ed8-9d9e-136ee013cf82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e99714658955413c8cb7134c04f39cf2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:44.370005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3777dacc-4e03-4ed8-9d9e-136ee013cf82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e99714658955413c8cb7134c04f39cf2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:44.370920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3777dacc-4e03-4ed8-9d9e-136ee013cf82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e99714658955413c8cb7134c04f39cf2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:44.393919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3777dacc-4e03-4ed8-9d9e-136ee013cf82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e99714658955413c8cb7134c04f39cf2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:44.488844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3777dacc-4e03-4ed8-9d9e-136ee013cf82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e99714658955413c8cb7134c04f39cf2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:44.541906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 831/1657] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:41:43 2022] DELETE /identity/v3/users/e99714658955413c8cb7134c04f39cf2 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:46.316317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4376c-a388-47be-a97c-05a604f8903f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:46.316317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4376c-a388-47be-a97c-05a604f8903f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:46.316317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4376c-a388-47be-a97c-05a604f8903f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:46.388597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-97e4376c-a388-47be-a97c-05a604f8903f None None] MFA Rules not processed for user `92fa59772a574f42b96a0776c269b93c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:46.390077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97e4376c-a388-47be-a97c-05a604f8903f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:46.546462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 827/1658] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:46.547965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd1fdc-8a3b-4dd3-b5be-e28dba64b09c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:46.548442 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd1fdc-8a3b-4dd3-b5be-e28dba64b09c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:46.549706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd1fdc-8a3b-4dd3-b5be-e28dba64b09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:46.575805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8fcd1fdc-8a3b-4dd3-b5be-e28dba64b09c None None] MFA Rules not processed for user `92fa59772a574f42b96a0776c269b93c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:46.577067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8fcd1fdc-8a3b-4dd3-b5be-e28dba64b09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:46.585402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 832/1659] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:46.599648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4376c-a388-47be-a97c-05a604f8903f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:46.601946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97e4376c-a388-47be-a97c-05a604f8903f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:46.619231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd1fdc-8a3b-4dd3-b5be-e28dba64b09c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:46.619231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8fcd1fdc-8a3b-4dd3-b5be-e28dba64b09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:46.682317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:46.731724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd1fdc-8a3b-4dd3-b5be-e28dba64b09c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:46.740711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:46.741461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:46.741748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:46.742026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] PATH_INFO: `/v3/projects/090cba77061046a3bd569ed7da54e2fc` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:46.744072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=090cba77061046a3bd569ed7da54e2fc)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:46.745121 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:41:46.745121 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Aug 04 07:41:46.745121 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: Aug 04 07:41:46.750500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:46.750500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:46.778725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4376c-a388-47be-a97c-05a604f8903f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:46.788085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09002766-3824-4f60-bb96-f962a10340bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:46.789028 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09002766-3824-4f60-bb96-f962a10340bc service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:46.790127 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09002766-3824-4f60-bb96-f962a10340bc service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:46.790610 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09002766-3824-4f60-bb96-f962a10340bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:46.817760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09002766-3824-4f60-bb96-f962a10340bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:46.819608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09002766-3824-4f60-bb96-f962a10340bc service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:46.832192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 828/1660] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:46.893238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090cba77061046a3bd569ed7da54e2fc', 'request_id': 'req-8d0961eb-4509-447a-83a4-362723850fac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:46.915798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090cba77061046a3bd569ed7da54e2fc', 'request_id': 'req-8d0961eb-4509-447a-83a4-362723850fac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:46.993882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090cba77061046a3bd569ed7da54e2fc', 'request_id': 'req-8d0961eb-4509-447a-83a4-362723850fac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:47.010513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] The token cache is being invalidate because project 090cba77061046a3bd569ed7da54e2fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:41:47.011213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:47.012288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8d0961eb-4509-447a-83a4-362723850fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:47.018694 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 833/1661] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:41:46 2022] DELETE /identity/v3/projects/090cba77061046a3bd569ed7da54e2fc => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:47.019931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa0ee7-72ba-42ab-b01f-5ea2fea4b606 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.020246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa0ee7-72ba-42ab-b01f-5ea2fea4b606 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.020532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa0ee7-72ba-42ab-b01f-5ea2fea4b606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.050732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8efa0ee7-72ba-42ab-b01f-5ea2fea4b606 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:47.052022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8efa0ee7-72ba-42ab-b01f-5ea2fea4b606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:47.134615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 829/1662] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:47.143188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6a658-c6c2-48da-9bff-02afbc1c5f1c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.143545 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6a658-c6c2-48da-9bff-02afbc1c5f1c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.147208 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6a658-c6c2-48da-9bff-02afbc1c5f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.170666 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1db6a658-c6c2-48da-9bff-02afbc1c5f1c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:47.172113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1db6a658-c6c2-48da-9bff-02afbc1c5f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:47.185403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 834/1663] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:47.187338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa0ee7-72ba-42ab-b01f-5ea2fea4b606 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:47.199314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa0ee7-72ba-42ab-b01f-5ea2fea4b606 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:47.205800 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b86dd-02f8-41de-9f9e-f30dac0f5bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:47.206575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b86dd-02f8-41de-9f9e-f30dac0f5bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.207000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b86dd-02f8-41de-9f9e-f30dac0f5bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.207340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b86dd-02f8-41de-9f9e-f30dac0f5bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.208315 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b86dd-02f8-41de-9f9e-f30dac0f5bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:47.208954 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b86dd-02f8-41de-9f9e-f30dac0f5bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:47.210635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b86dd-02f8-41de-9f9e-f30dac0f5bb2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:47.239245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 830/1664] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:41:47.241039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6a658-c6c2-48da-9bff-02afbc1c5f1c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:47.261155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6a658-c6c2-48da-9bff-02afbc1c5f1c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:47.264741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f34da8-d767-4906-91a4-7f539b8cc266 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.265225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f34da8-d767-4906-91a4-7f539b8cc266 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.265999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f34da8-d767-4906-91a4-7f539b8cc266 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.272046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 831/1665] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:41:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:41:47.275518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902be40-e693-4ab5-9d6b-492e15f352a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:47.276861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902be40-e693-4ab5-9d6b-492e15f352a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.277353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902be40-e693-4ab5-9d6b-492e15f352a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.278062 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902be40-e693-4ab5-9d6b-492e15f352a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.278928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902be40-e693-4ab5-9d6b-492e15f352a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:47.281360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902be40-e693-4ab5-9d6b-492e15f352a3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:47.310157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3164-3eab-49d7-adcc-6dde7f11814a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.311468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3164-3eab-49d7-adcc-6dde7f11814a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.311891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3164-3eab-49d7-adcc-6dde7f11814a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.378132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4e5a3164-3eab-49d7-adcc-6dde7f11814a None None] MFA Rules not processed for user `affaeff3cd294584b7238909e0457f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:47.381708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4e5a3164-3eab-49d7-adcc-6dde7f11814a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:47.403867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 835/1666] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:47 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:47.406006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76321aa0-5322-46e1-a886-2689d7b12fd7 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.406400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76321aa0-5322-46e1-a886-2689d7b12fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.406834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76321aa0-5322-46e1-a886-2689d7b12fd7 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.409702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 836/1667] 10.222.0.64 () {54 vars in 912 bytes} [Thu Aug 4 07:41:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:41:47.417075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76321aa0-5322-46e1-a886-2689d7b12fd7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:47.507250 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 832/1668] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:41:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:47.508665 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fba66c-8c4a-4950-851d-31623f92a0e1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.510226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fba66c-8c4a-4950-851d-31623f92a0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.510694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fba66c-8c4a-4950-851d-31623f92a0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.574288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76321aa0-5322-46e1-a886-2689d7b12fd7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:47.587726 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd54cc-022f-4c50-8920-58ff418c480f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:47.588428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd54cc-022f-4c50-8920-58ff418c480f admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.591053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd54cc-022f-4c50-8920-58ff418c480f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.591508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd54cc-022f-4c50-8920-58ff418c480f admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.592425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd54cc-022f-4c50-8920-58ff418c480f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:47.596634 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd54cc-022f-4c50-8920-58ff418c480f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:47.603713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-95fba66c-8c4a-4950-851d-31623f92a0e1 None None] MFA Rules not processed for user `b80e701a81cb4fa893e55288559010f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:47.608225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-95fba66c-8c4a-4950-851d-31623f92a0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:47.752186 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 837/1669] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:47 2022] POST /identity/v3/users => generated 287 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:47.757994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3367d-8dc8-43d8-ab56-0e10516c616b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.758294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3367d-8dc8-43d8-ab56-0e10516c616b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.758448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 833/1670] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:41:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:47.759883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b0313-3986-4c82-a5dc-53c28324453e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.760324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b0313-3986-4c82-a5dc-53c28324453e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.760725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b0313-3986-4c82-a5dc-53c28324453e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.761151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3367d-8dc8-43d8-ab56-0e10516c616b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.762857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 834/1671] 10.222.0.64 () {54 vars in 961 bytes} [Thu Aug 4 07:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:41:47.763515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b0313-3986-4c82-a5dc-53c28324453e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:47.780043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b0313-3986-4c82-a5dc-53c28324453e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:47.790834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e773eb-be4e-4c40-9547-e52a9d61a783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:47.791667 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e773eb-be4e-4c40-9547-e52a9d61a783 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.791907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e773eb-be4e-4c40-9547-e52a9d61a783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.792280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e773eb-be4e-4c40-9547-e52a9d61a783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.793096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e773eb-be4e-4c40-9547-e52a9d61a783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:47.794993 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e773eb-be4e-4c40-9547-e52a9d61a783 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:47.824133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 835/1672] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:47.830893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229aeba-a358-4995-beb3-9ee4dfc4c4c7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:47.833824 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229aeba-a358-4995-beb3-9ee4dfc4c4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:47.835163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229aeba-a358-4995-beb3-9ee4dfc4c4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:47.851252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3aa3367d-8dc8-43d8-ab56-0e10516c616b None None] MFA Rules not processed for user `e44a63323cfb40a2bb7a5e83d8636f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:47.854509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3aa3367d-8dc8-43d8-ab56-0e10516c616b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:47.885046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7229aeba-a358-4995-beb3-9ee4dfc4c4c7 None None] MFA Rules not processed for user `affaeff3cd294584b7238909e0457f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:47.886844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7229aeba-a358-4995-beb3-9ee4dfc4c4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:48.022313 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:48.043588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 836/1673] 10.222.0.64 () {58 vars in 1058 bytes} [Thu Aug 4 07:41:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:48.045923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229aeba-a358-4995-beb3-9ee4dfc4c4c7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:48.049142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7229aeba-a358-4995-beb3-9ee4dfc4c4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:48.083252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229aeba-a358-4995-beb3-9ee4dfc4c4c7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:48.099557 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e28563-ada0-40f1-8041-2c08ae810d1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:48.101466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e28563-ada0-40f1-8041-2c08ae810d1f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:48.103302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e28563-ada0-40f1-8041-2c08ae810d1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:48.104300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 838/1674] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:48.117065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e28563-ada0-40f1-8041-2c08ae810d1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:48.119295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3367d-8dc8-43d8-ab56-0e10516c616b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:48.138126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e28563-ada0-40f1-8041-2c08ae810d1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:48.140889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e28563-ada0-40f1-8041-2c08ae810d1f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:48.147657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3367d-8dc8-43d8-ab56-0e10516c616b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:48.161113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f44fb2-60ef-49a0-9cfa-1524b8256892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:48.161813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44fb2-60ef-49a0-9cfa-1524b8256892 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:48.163375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44fb2-60ef-49a0-9cfa-1524b8256892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:48.164356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 837/1675] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:48.164809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44fb2-60ef-49a0-9cfa-1524b8256892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:48.165551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f44fb2-60ef-49a0-9cfa-1524b8256892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:48.166110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e28563-ada0-40f1-8041-2c08ae810d1f service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:48.173362 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f44fb2-60ef-49a0-9cfa-1524b8256892 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:48.176199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:48.205652 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e28563-ada0-40f1-8041-2c08ae810d1f service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:48.206719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 839/1676] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:48.207918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f44fb2-60ef-49a0-9cfa-1524b8256892 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:48.212561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-53f44fb2-60ef-49a0-9cfa-1524b8256892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:48.215461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092ef68-04d9-44bb-8b9c-e0ac58af5951 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:48.216162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092ef68-04d9-44bb-8b9c-e0ac58af5951 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:48.217479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092ef68-04d9-44bb-8b9c-e0ac58af5951 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:48.217769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092ef68-04d9-44bb-8b9c-e0ac58af5951 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:48.245744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092ef68-04d9-44bb-8b9c-e0ac58af5951 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:48.247373 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092ef68-04d9-44bb-8b9c-e0ac58af5951 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:48.276451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 838/1677] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:48.287226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092ef68-04d9-44bb-8b9c-e0ac58af5951 service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:48.330549 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092ef68-04d9-44bb-8b9c-e0ac58af5951 service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:48.343310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c2315-4304-437c-ae87-7e465146473a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:48.352360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c2315-4304-437c-ae87-7e465146473a admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:48.359722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c2315-4304-437c-ae87-7e465146473a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:48.360199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c2315-4304-437c-ae87-7e465146473a admin admin] PATH_INFO: `/v3/projects/3e56d5c8fb794a74b232dc0aa3ab1a01/users/9604357fa56b40c1b83e291dfb193012/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:48.369658 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f44fb2-60ef-49a0-9cfa-1524b8256892 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:48.385605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447b7fa-d04f-4405-9d51-df55bdc0e47a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:48.386264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447b7fa-d04f-4405-9d51-df55bdc0e47a service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:48.386552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447b7fa-d04f-4405-9d51-df55bdc0e47a service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:48.386835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447b7fa-d04f-4405-9d51-df55bdc0e47a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:48.401650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c2315-4304-437c-ae87-7e465146473a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e56d5c8fb794a74b232dc0aa3ab1a01, user_id=9604357fa56b40c1b83e291dfb193012, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:48.402233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447b7fa-d04f-4405-9d51-df55bdc0e47a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:48.405197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c2315-4304-437c-ae87-7e465146473a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:48.405849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447b7fa-d04f-4405-9d51-df55bdc0e47a service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:48.423954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 840/1678] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:48.520265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 839/1679] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:48 2022] PUT /identity/v3/projects/3e56d5c8fb794a74b232dc0aa3ab1a01/users/9604357fa56b40c1b83e291dfb193012/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:48.521944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c44380-ce7d-4dc7-9d1f-90dab8f4905a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:48.522258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c44380-ce7d-4dc7-9d1f-90dab8f4905a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:48.522562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c44380-ce7d-4dc7-9d1f-90dab8f4905a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:48.601817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-99c44380-ce7d-4dc7-9d1f-90dab8f4905a None None] MFA Rules not processed for user `9604357fa56b40c1b83e291dfb193012`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:48.607825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-99c44380-ce7d-4dc7-9d1f-90dab8f4905a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:48.853644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 841/1680] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:41:48.899699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c2315-4304-437c-ae87-7e465146473a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:48.913700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-174c2315-4304-437c-ae87-7e465146473a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:49.036106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c2315-4304-437c-ae87-7e465146473a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:49.045433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff505c5-32c1-4949-91c4-18d977f9bb53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:49.046343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff505c5-32c1-4949-91c4-18d977f9bb53 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:49.046656 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff505c5-32c1-4949-91c4-18d977f9bb53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:49.046929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff505c5-32c1-4949-91c4-18d977f9bb53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:49.077766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff505c5-32c1-4949-91c4-18d977f9bb53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:49.080346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff505c5-32c1-4949-91c4-18d977f9bb53 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:49.085018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:49.109864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 840/1681] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:49.142155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c44380-ce7d-4dc7-9d1f-90dab8f4905a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:49.264145 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:49.322991 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c44380-ce7d-4dc7-9d1f-90dab8f4905a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:49.335838 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bd968-31d3-4deb-874c-cd2adaf2e8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:49.336696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bd968-31d3-4deb-874c-cd2adaf2e8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:49.337065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bd968-31d3-4deb-874c-cd2adaf2e8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:49.337246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bd968-31d3-4deb-874c-cd2adaf2e8f5 admin admin] PATH_INFO: `/v3/users/9604357fa56b40c1b83e291dfb193012` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:49.341611 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bd968-31d3-4deb-874c-cd2adaf2e8f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9604357fa56b40c1b83e291dfb193012)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:49.343567 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bd968-31d3-4deb-874c-cd2adaf2e8f5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:49.599954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-625bd968-31d3-4deb-874c-cd2adaf2e8f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9604357fa56b40c1b83e291dfb193012'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:49.601478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-625bd968-31d3-4deb-874c-cd2adaf2e8f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9604357fa56b40c1b83e291dfb193012'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:49.604688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-625bd968-31d3-4deb-874c-cd2adaf2e8f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9604357fa56b40c1b83e291dfb193012'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:49.629002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-625bd968-31d3-4deb-874c-cd2adaf2e8f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9604357fa56b40c1b83e291dfb193012'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:49.790041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-625bd968-31d3-4deb-874c-cd2adaf2e8f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9604357fa56b40c1b83e291dfb193012'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:49.816858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 842/1682] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:41:49 2022] DELETE /identity/v3/users/9604357fa56b40c1b83e291dfb193012 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:41:49.822271 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cf95d-f828-48ac-bd47-8952ee506039 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:49.823137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cf95d-f828-48ac-bd47-8952ee506039 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:49.823556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cf95d-f828-48ac-bd47-8952ee506039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:49.876806 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c73cf95d-f828-48ac-bd47-8952ee506039 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:49.880859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c73cf95d-f828-48ac-bd47-8952ee506039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:49.961363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0b5e3-38d6-44f6-a950-887fe6ae36cc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:49.961841 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0b5e3-38d6-44f6-a950-887fe6ae36cc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:49.962372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0b5e3-38d6-44f6-a950-887fe6ae36cc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:49.964266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 843/1683] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:41:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:41:50.027651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae2cd7-6f0a-4ea6-9ccf-97bb50bb493b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:50.028183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae2cd7-6f0a-4ea6-9ccf-97bb50bb493b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:50.028627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae2cd7-6f0a-4ea6-9ccf-97bb50bb493b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:50.067730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 841/1684] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:50.114076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cf95d-f828-48ac-bd47-8952ee506039 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:50.117491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c73cf95d-f828-48ac-bd47-8952ee506039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:50.121662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-45ae2cd7-6f0a-4ea6-9ccf-97bb50bb493b None None] MFA Rules not processed for user `affaeff3cd294584b7238909e0457f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:50.130396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-45ae2cd7-6f0a-4ea6-9ccf-97bb50bb493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:50.307533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 844/1685] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:41:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:41:50.308865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae2cd7-6f0a-4ea6-9ccf-97bb50bb493b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:50.311719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-45ae2cd7-6f0a-4ea6-9ccf-97bb50bb493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:50.315270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cf95d-f828-48ac-bd47-8952ee506039 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:50.345713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d45bb-d743-4844-a6a6-17b9178f9b67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:50.347028 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d45bb-d743-4844-a6a6-17b9178f9b67 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:50.347329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d45bb-d743-4844-a6a6-17b9178f9b67 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:50.347601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d45bb-d743-4844-a6a6-17b9178f9b67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:50.353617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e65d45bb-d743-4844-a6a6-17b9178f9b67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:50.391920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d45bb-d743-4844-a6a6-17b9178f9b67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:50.393596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d45bb-d743-4844-a6a6-17b9178f9b67 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:50.414737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 842/1686] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:41:50.421266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d45bb-d743-4844-a6a6-17b9178f9b67 service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:50.421266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e65d45bb-d743-4844-a6a6-17b9178f9b67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:50.443441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae2cd7-6f0a-4ea6-9ccf-97bb50bb493b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:50.447501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e0964-40dd-46cf-bbde-1cc4caf44836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:50.449959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e0964-40dd-46cf-bbde-1cc4caf44836 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:50.449959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e0964-40dd-46cf-bbde-1cc4caf44836 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:50.449959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e0964-40dd-46cf-bbde-1cc4caf44836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:50.460141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e0964-40dd-46cf-bbde-1cc4caf44836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:50.461880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e0964-40dd-46cf-bbde-1cc4caf44836 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:50.474822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d45bb-d743-4844-a6a6-17b9178f9b67 service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:50.493565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 845/1687] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:50.494653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e0964-40dd-46cf-bbde-1cc4caf44836 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:50.499537 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-426e0964-40dd-46cf-bbde-1cc4caf44836 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:50.508924 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:50.509636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:50.515071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:50.515421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] PATH_INFO: `/v3/users/811be94519f040e9abdc918dc3020567` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:50.524512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=811be94519f040e9abdc918dc3020567)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:50.525505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:50.530908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 07:41:50.530908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Aug 04 07:41:50.530908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: Aug 04 07:41:50.534524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:50.538894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e0964-40dd-46cf-bbde-1cc4caf44836 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:50.551539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6207092-283d-4c08-a1ab-64c0a1eb44ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:50.552499 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6207092-283d-4c08-a1ab-64c0a1eb44ff service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:50.553004 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6207092-283d-4c08-a1ab-64c0a1eb44ff service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:50.553415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6207092-283d-4c08-a1ab-64c0a1eb44ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:50.573761 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6207092-283d-4c08-a1ab-64c0a1eb44ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:50.583627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6207092-283d-4c08-a1ab-64c0a1eb44ff service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:50.620911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 846/1688] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:50.982420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '811be94519f040e9abdc918dc3020567'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:50.984800 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '811be94519f040e9abdc918dc3020567'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:51.021485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '811be94519f040e9abdc918dc3020567'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:51.086401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '811be94519f040e9abdc918dc3020567'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:51.170527 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '811be94519f040e9abdc918dc3020567'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:51.173380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:51.180878 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 843/1689] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:41:50 2022] DELETE /identity/v3/users/811be94519f040e9abdc918dc3020567 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:51.465829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6207092-283d-4c08-a1ab-64c0a1eb44ff service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:51.469308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b6207092-283d-4c08-a1ab-64c0a1eb44ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:51.504274 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6207092-283d-4c08-a1ab-64c0a1eb44ff service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:51.678900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:51.680819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:51.844248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:51.844248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:51.844248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:51.850079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] PATH_INFO: `/v3/projects/3e56d5c8fb794a74b232dc0aa3ab1a01` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:51.854191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e56d5c8fb794a74b232dc0aa3ab1a01)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:51.856197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:52.018017 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e03a35-f2e8-4ab7-83b8-3787b6fa8f6b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:52.035705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede308d-7b9f-4d8b-a51d-a7b722bfc866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:52.039684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede308d-7b9f-4d8b-a51d-a7b722bfc866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:52.040456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede308d-7b9f-4d8b-a51d-a7b722bfc866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:52.040886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede308d-7b9f-4d8b-a51d-a7b722bfc866 admin admin] PATH_INFO: `/v3/users/869b58008dd2449fbf31bafb8327fcb0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:52.044111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede308d-7b9f-4d8b-a51d-a7b722bfc866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=869b58008dd2449fbf31bafb8327fcb0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:52.047012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede308d-7b9f-4d8b-a51d-a7b722bfc866 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:52.112650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e56d5c8fb794a74b232dc0aa3ab1a01', 'request_id': 'req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.125218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e56d5c8fb794a74b232dc0aa3ab1a01', 'request_id': 'req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.240681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e56d5c8fb794a74b232dc0aa3ab1a01', 'request_id': 'req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.259969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] The token cache is being invalidate because project 3e56d5c8fb794a74b232dc0aa3ab1a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:41:52.260284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.267906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.270018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 847/1690] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:41:51 2022] DELETE /identity/v3/projects/3e56d5c8fb794a74b232dc0aa3ab1a01 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:52.277852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:52.291943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:52.327434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5ede308d-7b9f-4d8b-a51d-a7b722bfc866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '869b58008dd2449fbf31bafb8327fcb0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.328711 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5ede308d-7b9f-4d8b-a51d-a7b722bfc866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '869b58008dd2449fbf31bafb8327fcb0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.342044 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5ede308d-7b9f-4d8b-a51d-a7b722bfc866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '869b58008dd2449fbf31bafb8327fcb0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.369534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5ede308d-7b9f-4d8b-a51d-a7b722bfc866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '869b58008dd2449fbf31bafb8327fcb0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.440692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5ede308d-7b9f-4d8b-a51d-a7b722bfc866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '869b58008dd2449fbf31bafb8327fcb0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.450786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 844/1691] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:41:51 2022] DELETE /identity/v3/users/869b58008dd2449fbf31bafb8327fcb0 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:52.462797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661b5ef-f1e9-4af5-aaff-ae3488a97051 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:52.462797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661b5ef-f1e9-4af5-aaff-ae3488a97051 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:52.462797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661b5ef-f1e9-4af5-aaff-ae3488a97051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:52.493647 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dd59c-1d04-4dcc-b9bc-bdd10a17d268 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:52.507682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f661b5ef-f1e9-4af5-aaff-ae3488a97051 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:52.508994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f661b5ef-f1e9-4af5-aaff-ae3488a97051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:52.572796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:52.634887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:52.635663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:52.636057 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:52.636418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] PATH_INFO: `/v3/projects/4f9806b82f9e4854888cd294c7211cd1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:52.638426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f9806b82f9e4854888cd294c7211cd1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:52.650671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:52.696315 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 845/1692] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:52.699740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700e836-00a6-4c2d-b89a-0ed0707d0ec4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:52.699854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700e836-00a6-4c2d-b89a-0ed0707d0ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:52.702335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700e836-00a6-4c2d-b89a-0ed0707d0ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:52.749750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d700e836-00a6-4c2d-b89a-0ed0707d0ec4 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:52.751077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d700e836-00a6-4c2d-b89a-0ed0707d0ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:52.776712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 846/1693] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:52.776712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb6cb7-b0a3-48df-8ae8-5806e0abe2c0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:52.776712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb6cb7-b0a3-48df-8ae8-5806e0abe2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:52.776712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb6cb7-b0a3-48df-8ae8-5806e0abe2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:52.864116 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d8bb6cb7-b0a3-48df-8ae8-5806e0abe2c0 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:52.867492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d8bb6cb7-b0a3-48df-8ae8-5806e0abe2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:52.883435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f9806b82f9e4854888cd294c7211cd1', 'request_id': 'req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.897058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f9806b82f9e4854888cd294c7211cd1', 'request_id': 'req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:52.915846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:53.009001 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f9806b82f9e4854888cd294c7211cd1', 'request_id': 'req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:53.031715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] The token cache is being invalidate because project 4f9806b82f9e4854888cd294c7211cd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:41:53.037629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:53.039460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:53.042657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 848/1694] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:41:52 2022] DELETE /identity/v3/projects/4f9806b82f9e4854888cd294c7211cd1 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:53.048982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:53.058434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:53.073160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 847/1695] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:53.073160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb6cb7-b0a3-48df-8ae8-5806e0abe2c0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:53.073160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d8bb6cb7-b0a3-48df-8ae8-5806e0abe2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:53.112146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb6cb7-b0a3-48df-8ae8-5806e0abe2c0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:53.142076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:53.144645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:53.149757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:53.154373 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] PATH_INFO: `/v3/users/1272d313c8f34f42bfa34e245471a2fb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:53.163252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1272d313c8f34f42bfa34e245471a2fb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:53.169773 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:53.185964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d57ac-d16b-45eb-8ab0-31b298e5f691 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:53.207856 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4f31b-2d60-4b76-86b9-fd1d70cf51c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:53.209440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4f31b-2d60-4b76-86b9-fd1d70cf51c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:53.209440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4f31b-2d60-4b76-86b9-fd1d70cf51c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:53.209728 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4f31b-2d60-4b76-86b9-fd1d70cf51c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:53.215853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9cd4f31b-2d60-4b76-86b9-fd1d70cf51c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:53.252212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4f31b-2d60-4b76-86b9-fd1d70cf51c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:53.253908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4f31b-2d60-4b76-86b9-fd1d70cf51c9 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:53.283816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 849/1696] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:53.288291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4f31b-2d60-4b76-86b9-fd1d70cf51c9 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:53.335754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4f31b-2d60-4b76-86b9-fd1d70cf51c9 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:53.359745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc52ed2-a766-4a0c-af35-254753f784e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:53.361980 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc52ed2-a766-4a0c-af35-254753f784e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:53.362511 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc52ed2-a766-4a0c-af35-254753f784e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:53.362964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc52ed2-a766-4a0c-af35-254753f784e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:53.364926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc52ed2-a766-4a0c-af35-254753f784e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:53.365416 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc52ed2-a766-4a0c-af35-254753f784e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:53.367689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc52ed2-a766-4a0c-af35-254753f784e0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:53.391516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 850/1697] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:41:53.393587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc52ed2-a766-4a0c-af35-254753f784e0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:53.407810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc52ed2-a766-4a0c-af35-254753f784e0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:53.415768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9024d-f565-40df-94d1-6a02f0138a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:53.423625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9024d-f565-40df-94d1-6a02f0138a6b admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:53.423625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9024d-f565-40df-94d1-6a02f0138a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:53.423625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9024d-f565-40df-94d1-6a02f0138a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:53.423625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9024d-f565-40df-94d1-6a02f0138a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:53.423625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9024d-f565-40df-94d1-6a02f0138a6b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:53.504883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1272d313c8f34f42bfa34e245471a2fb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:53.509868 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1272d313c8f34f42bfa34e245471a2fb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:53.540999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1272d313c8f34f42bfa34e245471a2fb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:53.572985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 851/1698] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:53 2022] POST /identity/v3/projects => generated 375 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:53.578271 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1272d313c8f34f42bfa34e245471a2fb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:53.583129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9024d-f565-40df-94d1-6a02f0138a6b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:53.594241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-31f9024d-f565-40df-94d1-6a02f0138a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:53.671739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1272d313c8f34f42bfa34e245471a2fb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:53.683241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 848/1699] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:41:53 2022] DELETE /identity/v3/users/1272d313c8f34f42bfa34e245471a2fb => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:53.779652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9024d-f565-40df-94d1-6a02f0138a6b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:53.974479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abd05a-1e1a-43ee-9edb-5b6e199ef2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:53.980229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abd05a-1e1a-43ee-9edb-5b6e199ef2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:53.980589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abd05a-1e1a-43ee-9edb-5b6e199ef2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:53.980868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abd05a-1e1a-43ee-9edb-5b6e199ef2b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:53.982015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abd05a-1e1a-43ee-9edb-5b6e199ef2b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:53.986505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abd05a-1e1a-43ee-9edb-5b6e199ef2b9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:54.119802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 852/1700] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:53 2022] POST /identity/v3/users => generated 288 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:54.120387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:54.132798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03ca17-6056-4fb4-ab44-4268c2ae2074 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:54.141577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541d674-701d-4d23-8964-ef84f3bef5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:54.146109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541d674-701d-4d23-8964-ef84f3bef5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:54.146109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541d674-701d-4d23-8964-ef84f3bef5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:54.146109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541d674-701d-4d23-8964-ef84f3bef5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:54.146109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541d674-701d-4d23-8964-ef84f3bef5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:54.146109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541d674-701d-4d23-8964-ef84f3bef5ad admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:54.161976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 849/1701] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:54.167030 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abd05a-1e1a-43ee-9edb-5b6e199ef2b9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:54.181300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abd05a-1e1a-43ee-9edb-5b6e199ef2b9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:54.191886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629a57d-13e9-4214-900a-0460bfcbff7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:54.193729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629a57d-13e9-4214-900a-0460bfcbff7e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:54.194795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629a57d-13e9-4214-900a-0460bfcbff7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:54.195397 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629a57d-13e9-4214-900a-0460bfcbff7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:54.196400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629a57d-13e9-4214-900a-0460bfcbff7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:54.198957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629a57d-13e9-4214-900a-0460bfcbff7e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:54.213571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 853/1702] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:54.218039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541d674-701d-4d23-8964-ef84f3bef5ad admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:54.230602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541d674-701d-4d23-8964-ef84f3bef5ad admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:54.237069 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386915e-f27a-4f66-9768-b8f11f159278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:54.238091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386915e-f27a-4f66-9768-b8f11f159278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:54.238588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386915e-f27a-4f66-9768-b8f11f159278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:54.239139 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386915e-f27a-4f66-9768-b8f11f159278 admin admin] PATH_INFO: `/v3/projects/d921271c706843de875563f45467c9c8/users/65e1af5780284e60915b839e172eb7b5/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:54.260341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386915e-f27a-4f66-9768-b8f11f159278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d921271c706843de875563f45467c9c8, user_id=65e1af5780284e60915b839e172eb7b5, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:54.264264 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386915e-f27a-4f66-9768-b8f11f159278 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:54.268631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629a57d-13e9-4214-900a-0460bfcbff7e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:54.270838 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e629a57d-13e9-4214-900a-0460bfcbff7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:54.292574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629a57d-13e9-4214-900a-0460bfcbff7e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:54.307207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8a9dc-4f05-4f74-a9cf-75f7f5c2db99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:54.307896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a9dc-4f05-4f74-a9cf-75f7f5c2db99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:54.308211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a9dc-4f05-4f74-a9cf-75f7f5c2db99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:54.308527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a9dc-4f05-4f74-a9cf-75f7f5c2db99 admin admin] PATH_INFO: `/v3/users/f7a5713174a74031b0d92610df1e28ef` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:54.310754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a9dc-4f05-4f74-a9cf-75f7f5c2db99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7a5713174a74031b0d92610df1e28ef)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:54.312780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a9dc-4f05-4f74-a9cf-75f7f5c2db99 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:54.364991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 850/1703] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:54 2022] PUT /identity/v3/projects/d921271c706843de875563f45467c9c8/users/65e1af5780284e60915b839e172eb7b5/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:54.367124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c293be-c516-4531-9ac5-03fbada7e494 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:54.367424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c293be-c516-4531-9ac5-03fbada7e494 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:54.367708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c293be-c516-4531-9ac5-03fbada7e494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:54.397950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a5c293be-c516-4531-9ac5-03fbada7e494 None None] MFA Rules not processed for user `65e1af5780284e60915b839e172eb7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:54.399216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5c293be-c516-4531-9ac5-03fbada7e494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:54.552304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 851/1704] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:54.572307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c293be-c516-4531-9ac5-03fbada7e494 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:54.577431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5c293be-c516-4531-9ac5-03fbada7e494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:54.587746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-62a8a9dc-4f05-4f74-a9cf-75f7f5c2db99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7a5713174a74031b0d92610df1e28ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:54.590250 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:54.591178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-62a8a9dc-4f05-4f74-a9cf-75f7f5c2db99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7a5713174a74031b0d92610df1e28ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:54.593311 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-62a8a9dc-4f05-4f74-a9cf-75f7f5c2db99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7a5713174a74031b0d92610df1e28ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:54.624058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-62a8a9dc-4f05-4f74-a9cf-75f7f5c2db99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7a5713174a74031b0d92610df1e28ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:54.722021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c293be-c516-4531-9ac5-03fbada7e494 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:54.726714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0d8e5-1b3e-4f7e-b43b-f0f1fba77ee8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfc4dfda424d40b7c9f17345b8ad45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:54.727456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0d8e5-1b3e-4f7e-b43b-f0f1fba77ee8 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:54.727940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0d8e5-1b3e-4f7e-b43b-f0f1fba77ee8 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:54.728335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0d8e5-1b3e-4f7e-b43b-f0f1fba77ee8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:54.730461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f3b0d8e5-1b3e-4f7e-b43b-f0f1fba77ee8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:54.737510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-62a8a9dc-4f05-4f74-a9cf-75f7f5c2db99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7a5713174a74031b0d92610df1e28ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:54.764214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 854/1705] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:41:54 2022] DELETE /identity/v3/users/f7a5713174a74031b0d92610df1e28ef => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:41:54.764442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0d8e5-1b3e-4f7e-b43b-f0f1fba77ee8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:54.764442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0d8e5-1b3e-4f7e-b43b-f0f1fba77ee8 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:54.766492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a8d1f-af51-47ca-96f8-190175ba4837 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:54.766858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a8d1f-af51-47ca-96f8-190175ba4837 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:54.767304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a8d1f-af51-47ca-96f8-190175ba4837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:54.807973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:54.821739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 852/1706] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:54.850970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1f3a8d1f-af51-47ca-96f8-190175ba4837 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:54.853232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1f3a8d1f-af51-47ca-96f8-190175ba4837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:54.968714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0d8e5-1b3e-4f7e-b43b-f0f1fba77ee8 service glance] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:54.971001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f3b0d8e5-1b3e-4f7e-b43b-f0f1fba77ee8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:55.008028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 855/1707] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:55.018514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a8d1f-af51-47ca-96f8-190175ba4837 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:55.020505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1f3a8d1f-af51-47ca-96f8-190175ba4837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:55.131862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a8d1f-af51-47ca-96f8-190175ba4837 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:55.150157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332837e-60d7-4fe3-98ac-ac274cb039d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:55.150813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332837e-60d7-4fe3-98ac-ac274cb039d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:55.151102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332837e-60d7-4fe3-98ac-ac274cb039d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:55.151383 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332837e-60d7-4fe3-98ac-ac274cb039d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:55.188479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0d8e5-1b3e-4f7e-b43b-f0f1fba77ee8 service glance] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:55.206090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:55.206476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0332837e-60d7-4fe3-98ac-ac274cb039d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:55.207159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:55.207587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:55.207955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0332837e-60d7-4fe3-98ac-ac274cb039d4 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:55.208452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] PATH_INFO: `/v3/projects/3816f649c4c64f0ea9c0c5be8780794c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:55.213686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3816f649c4c64f0ea9c0c5be8780794c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:55.216068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:55.246940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 856/1708] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:41:55.415507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3816f649c4c64f0ea9c0c5be8780794c', 'request_id': 'req-e54fd168-3328-4996-aab5-50245152a09d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:55.505060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3816f649c4c64f0ea9c0c5be8780794c', 'request_id': 'req-e54fd168-3328-4996-aab5-50245152a09d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:55.514834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3816f649c4c64f0ea9c0c5be8780794c', 'request_id': 'req-e54fd168-3328-4996-aab5-50245152a09d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:55.559370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] The token cache is being invalidate because project 3816f649c4c64f0ea9c0c5be8780794c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:41:55.559710 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:55.562619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e54fd168-3328-4996-aab5-50245152a09d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:55.569137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 853/1709] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:41:54 2022] DELETE /identity/v3/projects/3816f649c4c64f0ea9c0c5be8780794c => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:55.569973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b64a3b-2871-4d55-b016-2091d994d1c0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:55.570572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b64a3b-2871-4d55-b016-2091d994d1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:55.570964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b64a3b-2871-4d55-b016-2091d994d1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:55.608983 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e4b64a3b-2871-4d55-b016-2091d994d1c0 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:55.610969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e4b64a3b-2871-4d55-b016-2091d994d1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:55.716251 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 857/1710] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:55.721210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee35d0-d918-40ef-8386-e8af2fc900df None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:55.722312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee35d0-d918-40ef-8386-e8af2fc900df None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:55.722596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee35d0-d918-40ef-8386-e8af2fc900df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:55.746448 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d863c9-c8ae-4231-ba94-40d7fb5f74a1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:55.746807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d863c9-c8ae-4231-ba94-40d7fb5f74a1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:55.747101 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d863c9-c8ae-4231-ba94-40d7fb5f74a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:55.758426 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-73ee35d0-d918-40ef-8386-e8af2fc900df None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:55.763437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-73ee35d0-d918-40ef-8386-e8af2fc900df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:55.786735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d4d863c9-c8ae-4231-ba94-40d7fb5f74a1 None None] MFA Rules not processed for user `88d78d9c0369431d88ad85c960f6a0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:55.790180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 854/1711] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:55.791031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee35d0-d918-40ef-8386-e8af2fc900df None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:55.792756 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d4d863c9-c8ae-4231-ba94-40d7fb5f74a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:55.802366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee35d0-d918-40ef-8386-e8af2fc900df None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:55.808715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955053b6-ca91-4c07-a9a9-3cba21d18300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:55.809925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955053b6-ca91-4c07-a9a9-3cba21d18300 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:55.810324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955053b6-ca91-4c07-a9a9-3cba21d18300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:55.810609 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955053b6-ca91-4c07-a9a9-3cba21d18300 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:55.811656 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955053b6-ca91-4c07-a9a9-3cba21d18300 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:55.811969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955053b6-ca91-4c07-a9a9-3cba21d18300 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:55.813930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955053b6-ca91-4c07-a9a9-3cba21d18300 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:55.847570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 855/1712] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:41:55.848870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955053b6-ca91-4c07-a9a9-3cba21d18300 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:55.852089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-955053b6-ca91-4c07-a9a9-3cba21d18300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:55.884442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955053b6-ca91-4c07-a9a9-3cba21d18300 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:55.893974 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:55.894716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:55.895077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:55.895445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] PATH_INFO: `/v3/projects/f555268a8f9442fe94f41f8ad411ec04` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:55.897709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f555268a8f9442fe94f41f8ad411ec04)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:55.902162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:55.963220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 858/1713] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:41:55.964415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d863c9-c8ae-4231-ba94-40d7fb5f74a1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:55.987824 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d863c9-c8ae-4231-ba94-40d7fb5f74a1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:55.993236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8e6d3-6d8b-4f4f-9023-15f0bcee70a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:55.993947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8e6d3-6d8b-4f4f-9023-15f0bcee70a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:55.994235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8e6d3-6d8b-4f4f-9023-15f0bcee70a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:55.994583 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8e6d3-6d8b-4f4f-9023-15f0bcee70a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:55.995182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8e6d3-6d8b-4f4f-9023-15f0bcee70a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:55.997995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8e6d3-6d8b-4f4f-9023-15f0bcee70a8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:56.073387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f555268a8f9442fe94f41f8ad411ec04', 'request_id': 'req-9c55c946-44d6-4af1-9b67-224a6122edc8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:56.157005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 859/1714] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:55 2022] POST /identity/v3/projects => generated 371 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:56.158112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3b8a-9034-42a7-be0c-f731dec12f78 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:56.159277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3b8a-9034-42a7-be0c-f731dec12f78 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:56.159277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3b8a-9034-42a7-be0c-f731dec12f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:56.188154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f555268a8f9442fe94f41f8ad411ec04', 'request_id': 'req-9c55c946-44d6-4af1-9b67-224a6122edc8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:56.198850 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f555268a8f9442fe94f41f8ad411ec04', 'request_id': 'req-9c55c946-44d6-4af1-9b67-224a6122edc8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:56.201885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-132e3b8a-9034-42a7-be0c-f731dec12f78 None None] MFA Rules not processed for user `88d78d9c0369431d88ad85c960f6a0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:56.203352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-132e3b8a-9034-42a7-be0c-f731dec12f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:56.225396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] The token cache is being invalidate because project f555268a8f9442fe94f41f8ad411ec04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:41:56.225689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:56.226521 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9c55c946-44d6-4af1-9b67-224a6122edc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:56.228793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 856/1715] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:41:55 2022] DELETE /identity/v3/projects/f555268a8f9442fe94f41f8ad411ec04 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:56.230210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba07186-5563-437a-aa7b-52a1fd1915c5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:56.230630 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba07186-5563-437a-aa7b-52a1fd1915c5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:56.231205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba07186-5563-437a-aa7b-52a1fd1915c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:56.276475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 860/1716] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:56.277333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e3b8a-9034-42a7-be0c-f731dec12f78 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:56.278714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-132e3b8a-9034-42a7-be0c-f731dec12f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:56.314075 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cba07186-5563-437a-aa7b-52a1fd1915c5 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:56.315377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cba07186-5563-437a-aa7b-52a1fd1915c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:56.390853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e3b8a-9034-42a7-be0c-f731dec12f78 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:56.404088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36fa64-9470-4b88-9d2d-60b235711f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:56.406424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 857/1717] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:41:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:56.417627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36fa64-9470-4b88-9d2d-60b235711f6c admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:56.417905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36fa64-9470-4b88-9d2d-60b235711f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:56.418195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36fa64-9470-4b88-9d2d-60b235711f6c admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:56.418939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36fa64-9470-4b88-9d2d-60b235711f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:56.420484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25439c4a-9f37-4c8d-8264-1392e3a5c19f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:56.423972 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25439c4a-9f37-4c8d-8264-1392e3a5c19f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:56.424524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25439c4a-9f37-4c8d-8264-1392e3a5c19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:56.430068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36fa64-9470-4b88-9d2d-60b235711f6c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:56.469473 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-25439c4a-9f37-4c8d-8264-1392e3a5c19f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:56.472371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-25439c4a-9f37-4c8d-8264-1392e3a5c19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:56.483913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 858/1718] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:56.485070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25439c4a-9f37-4c8d-8264-1392e3a5c19f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:56.486996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-25439c4a-9f37-4c8d-8264-1392e3a5c19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:56.557361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 861/1719] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:56 2022] POST /identity/v3/users => generated 286 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:56.581631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36fa64-9470-4b88-9d2d-60b235711f6c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:56.585996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ee36fa64-9470-4b88-9d2d-60b235711f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:56.624079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36fa64-9470-4b88-9d2d-60b235711f6c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:56.635931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:56.645172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:56.645712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:56.647520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] PATH_INFO: `/v3/projects/d9a525f6d21a4f708cfc945db5e57f12` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:56.650018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a525f6d21a4f708cfc945db5e57f12)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:56.654181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:56.707129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25439c4a-9f37-4c8d-8264-1392e3a5c19f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:56.717969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710ddd-c590-4b83-9952-f05737e60343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:56.725118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710ddd-c590-4b83-9952-f05737e60343 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:56.725118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710ddd-c590-4b83-9952-f05737e60343 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:56.725364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710ddd-c590-4b83-9952-f05737e60343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:56.728791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-81710ddd-c590-4b83-9952-f05737e60343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:56.753861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81710ddd-c590-4b83-9952-f05737e60343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:56.753861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81710ddd-c590-4b83-9952-f05737e60343 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:56.768774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 859/1720] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:56.782506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685747a-f7d1-4a50-882c-bfec1b84cce8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:56.782506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685747a-f7d1-4a50-882c-bfec1b84cce8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:56.782858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685747a-f7d1-4a50-882c-bfec1b84cce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:56.805463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a525f6d21a4f708cfc945db5e57f12', 'request_id': 'req-b08467e4-db2d-4be0-af35-120f238480fd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:56.816746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a525f6d21a4f708cfc945db5e57f12', 'request_id': 'req-b08467e4-db2d-4be0-af35-120f238480fd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:56.831324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b685747a-f7d1-4a50-882c-bfec1b84cce8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:56.834374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b685747a-f7d1-4a50-882c-bfec1b84cce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:56.880513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:56.900532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a525f6d21a4f708cfc945db5e57f12', 'request_id': 'req-b08467e4-db2d-4be0-af35-120f238480fd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:56.922044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] The token cache is being invalidate because project d9a525f6d21a4f708cfc945db5e57f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:41:56.922369 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:56.925342 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:56.933428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 862/1721] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:41:56 2022] DELETE /identity/v3/projects/d9a525f6d21a4f708cfc945db5e57f12 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:56.939305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:56.939305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:56.963357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:56.994973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 860/1722] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:56.996066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685747a-f7d1-4a50-882c-bfec1b84cce8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:56.999108 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b685747a-f7d1-4a50-882c-bfec1b84cce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:57.116224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08467e4-db2d-4be0-af35-120f238480fd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:57.133281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ce044-f698-4519-b96c-ece2bfec4edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:57.133931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ce044-f698-4519-b96c-ece2bfec4edd admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:57.134218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ce044-f698-4519-b96c-ece2bfec4edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:57.134502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ce044-f698-4519-b96c-ece2bfec4edd admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:57.135208 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ce044-f698-4519-b96c-ece2bfec4edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:57.140298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ce044-f698-4519-b96c-ece2bfec4edd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:57.159058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 863/1723] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:57.159058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ce044-f698-4519-b96c-ece2bfec4edd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:57.162199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8e8ce044-f698-4519-b96c-ece2bfec4edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:57.181823 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ce044-f698-4519-b96c-ece2bfec4edd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:57.187896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:57.189117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:57.190115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:57.191458 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] PATH_INFO: `/v3/users/6404342fec824715b6cedd8926132163` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:57.208897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6404342fec824715b6cedd8926132163)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:57.208897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:57.211253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685747a-f7d1-4a50-882c-bfec1b84cce8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:57.212657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f3179-864b-495a-86a6-1077eeb1d263 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:57.214480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f3179-864b-495a-86a6-1077eeb1d263 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:57.214927 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f3179-864b-495a-86a6-1077eeb1d263 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:57.215784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f3179-864b-495a-86a6-1077eeb1d263 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:57.218213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c91f3179-864b-495a-86a6-1077eeb1d263 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:57.399491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f3179-864b-495a-86a6-1077eeb1d263 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:57.401708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f3179-864b-495a-86a6-1077eeb1d263 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:57.427234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 861/1724] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:41:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:41:57.432797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df347f2e-785d-42d2-961d-9d5801d8c506 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:57.433182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df347f2e-785d-42d2-961d-9d5801d8c506 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:57.433583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df347f2e-785d-42d2-961d-9d5801d8c506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:57.472450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-df347f2e-785d-42d2-961d-9d5801d8c506 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:57.474065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-df347f2e-785d-42d2-961d-9d5801d8c506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:57.484568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 862/1725] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:41:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:57.487253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df347f2e-785d-42d2-961d-9d5801d8c506 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:57.489588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6404342fec824715b6cedd8926132163'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:57.492763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6404342fec824715b6cedd8926132163'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:57.495136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6404342fec824715b6cedd8926132163'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:57.509786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df347f2e-785d-42d2-961d-9d5801d8c506 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:57.518754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b5e1b-fae2-408d-aba7-ddce3e6d5614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:57.518754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b5e1b-fae2-408d-aba7-ddce3e6d5614 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:57.518754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b5e1b-fae2-408d-aba7-ddce3e6d5614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:57.519164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b5e1b-fae2-408d-aba7-ddce3e6d5614 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:57.520376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b5e1b-fae2-408d-aba7-ddce3e6d5614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:41:57.521039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b5e1b-fae2-408d-aba7-ddce3e6d5614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:57.523112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b5e1b-fae2-408d-aba7-ddce3e6d5614 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:57.556443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 863/1726] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:41:57.558561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b5e1b-fae2-408d-aba7-ddce3e6d5614 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:57.562133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-450b5e1b-fae2-408d-aba7-ddce3e6d5614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:57.575480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6404342fec824715b6cedd8926132163'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:57.583529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b5e1b-fae2-408d-aba7-ddce3e6d5614 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:57.590609 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811372c9-d028-4bc2-abbd-fd93e4689ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:57.592758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811372c9-d028-4bc2-abbd-fd93e4689ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:57.592758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811372c9-d028-4bc2-abbd-fd93e4689ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:57.592758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811372c9-d028-4bc2-abbd-fd93e4689ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:57.593482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811372c9-d028-4bc2-abbd-fd93e4689ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:57.599897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811372c9-d028-4bc2-abbd-fd93e4689ca2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:57.645910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 864/1727] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:57.656192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811372c9-d028-4bc2-abbd-fd93e4689ca2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:57.656192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-811372c9-d028-4bc2-abbd-fd93e4689ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:57.691791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811372c9-d028-4bc2-abbd-fd93e4689ca2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:57.707871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357365e3-3b34-4aa8-9437-426c9ba30d69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:57.709198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6404342fec824715b6cedd8926132163'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:57.709956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357365e3-3b34-4aa8-9437-426c9ba30d69 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:57.713478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357365e3-3b34-4aa8-9437-426c9ba30d69 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:57.713478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357365e3-3b34-4aa8-9437-426c9ba30d69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:57.718905 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-357365e3-3b34-4aa8-9437-426c9ba30d69 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:57.740440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 864/1728] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:41:57 2022] DELETE /identity/v3/users/6404342fec824715b6cedd8926132163 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:57.754472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:57.759461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:57.789002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357365e3-3b34-4aa8-9437-426c9ba30d69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:57.792035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357365e3-3b34-4aa8-9437-426c9ba30d69 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:57.811902 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:57.834851 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 865/1729] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:41:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:41:57.837830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357365e3-3b34-4aa8-9437-426c9ba30d69 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:57.865192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357365e3-3b34-4aa8-9437-426c9ba30d69 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:57.894489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:57.998338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00510893-9d29-4481-855e-9b09f46663d6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:58.007226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:58.022457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9995d23-bdde-4756-974c-90ef0d2a7438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:58.023264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9995d23-bdde-4756-974c-90ef0d2a7438 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:58.023646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9995d23-bdde-4756-974c-90ef0d2a7438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:58.024122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9995d23-bdde-4756-974c-90ef0d2a7438 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:58.024929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9995d23-bdde-4756-974c-90ef0d2a7438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:58.027237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9995d23-bdde-4756-974c-90ef0d2a7438 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:58.071272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b91270-74b2-41d2-997d-b0904b2544bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:58.072044 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b91270-74b2-41d2-997d-b0904b2544bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:58.072275 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b91270-74b2-41d2-997d-b0904b2544bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:58.072593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b91270-74b2-41d2-997d-b0904b2544bb admin admin] PATH_INFO: `/v3/projects/1c7bb7290e8e444faa0e82556b45d0d9/users/ba98b5d8e4b84af99668d8ffe7e9d047/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:58.134738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b91270-74b2-41d2-997d-b0904b2544bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7bb7290e8e444faa0e82556b45d0d9, user_id=ba98b5d8e4b84af99668d8ffe7e9d047, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:58.146499 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b91270-74b2-41d2-997d-b0904b2544bb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:58.206138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 865/1730] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:57 2022] POST /identity/v3/projects => generated 355 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:58.209631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ddeb0-5592-4fe1-806d-8ae14c1d090b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:58.213048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ddeb0-5592-4fe1-806d-8ae14c1d090b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:58.213454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ddeb0-5592-4fe1-806d-8ae14c1d090b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:58.292311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 866/1731] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:41:57 2022] PUT /identity/v3/projects/1c7bb7290e8e444faa0e82556b45d0d9/users/ba98b5d8e4b84af99668d8ffe7e9d047/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:58.302190 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b91270-74b2-41d2-997d-b0904b2544bb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:58.402988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5a5ddeb0-5592-4fe1-806d-8ae14c1d090b None None] MFA Rules not processed for user `5509519254574e50a70c84ae895acd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:58.408588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5a5ddeb0-5592-4fe1-806d-8ae14c1d090b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:58.632547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b91270-74b2-41d2-997d-b0904b2544bb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:58.670167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7796f4-cfbb-4457-9698-d14786ba74a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:58.671298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7796f4-cfbb-4457-9698-d14786ba74a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:58.671584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7796f4-cfbb-4457-9698-d14786ba74a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:58.671862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7796f4-cfbb-4457-9698-d14786ba74a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:58.675654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7796f4-cfbb-4457-9698-d14786ba74a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:58.678549 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 866/1732] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:58.685042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374cb0c-78d7-42d4-baf1-3d93890f6ec7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:58.685340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374cb0c-78d7-42d4-baf1-3d93890f6ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:58.689837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374cb0c-78d7-42d4-baf1-3d93890f6ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:58.689991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7796f4-cfbb-4457-9698-d14786ba74a8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:58.714801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:58.743831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9374cb0c-78d7-42d4-baf1-3d93890f6ec7 None None] MFA Rules not processed for user `ba98b5d8e4b84af99668d8ffe7e9d047`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:58.745985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9374cb0c-78d7-42d4-baf1-3d93890f6ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:58.811068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 867/1733] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:58 2022] POST /identity/v3/users => generated 278 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:58.811068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84329184-a75b-4297-8697-adce6255cbb0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:58.811068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84329184-a75b-4297-8697-adce6255cbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:58.811068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84329184-a75b-4297-8697-adce6255cbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:58.820464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:58.928066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-84329184-a75b-4297-8697-adce6255cbb0 None None] MFA Rules not processed for user `e44a63323cfb40a2bb7a5e83d8636f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:58.939708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-84329184-a75b-4297-8697-adce6255cbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:59.059900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 867/1734] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:59.059900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374cb0c-78d7-42d4-baf1-3d93890f6ec7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:59.059900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9374cb0c-78d7-42d4-baf1-3d93890f6ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:59.220383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 868/1735] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:41:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:59.223429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:41:59.225074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84329184-a75b-4297-8697-adce6255cbb0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:59.277451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84329184-a75b-4297-8697-adce6255cbb0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:59.285698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374cb0c-78d7-42d4-baf1-3d93890f6ec7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:59.294346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b673c5-11f2-405d-b470-d230f1e25843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:59.294346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b673c5-11f2-405d-b470-d230f1e25843 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:59.294346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b673c5-11f2-405d-b470-d230f1e25843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:59.294346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b673c5-11f2-405d-b470-d230f1e25843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:59.295024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b673c5-11f2-405d-b470-d230f1e25843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:59.296350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b673c5-11f2-405d-b470-d230f1e25843 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:59.308896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ef13f-89b7-4da2-aab3-c09298c0cbdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:59.309589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ef13f-89b7-4da2-aab3-c09298c0cbdc service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:59.309897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ef13f-89b7-4da2-aab3-c09298c0cbdc service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:59.310249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ef13f-89b7-4da2-aab3-c09298c0cbdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:59.348365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 869/1736] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:41:59.350020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b673c5-11f2-405d-b470-d230f1e25843 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:59.351755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ef13f-89b7-4da2-aab3-c09298c0cbdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:59.355338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f4b673c5-11f2-405d-b470-d230f1e25843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:59.359352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ef13f-89b7-4da2-aab3-c09298c0cbdc service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:59.392973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 868/1737] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:41:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:41:59.394423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ef13f-89b7-4da2-aab3-c09298c0cbdc service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:59.396819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b673c5-11f2-405d-b470-d230f1e25843 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:59.425403 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:59.427942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:59.429184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:59.429881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] PATH_INFO: `/v3/projects/f8f739e8d70a4a69971c92a665fce2cc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:59.431170 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ef13f-89b7-4da2-aab3-c09298c0cbdc service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:59.434302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8f739e8d70a4a69971c92a665fce2cc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:59.445249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:59.455003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53825c7b-1e7b-48ca-9454-2482aebd6d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:59.458978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53825c7b-1e7b-48ca-9454-2482aebd6d21 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:59.459374 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53825c7b-1e7b-48ca-9454-2482aebd6d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:59.459906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53825c7b-1e7b-48ca-9454-2482aebd6d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:59.460830 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53825c7b-1e7b-48ca-9454-2482aebd6d21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:59.466119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53825c7b-1e7b-48ca-9454-2482aebd6d21 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:41:59.664014 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 869/1738] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:41:59 2022] POST /identity/v3/projects => generated 373 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:41:59.665641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a123259-2dff-4e59-bf8d-2f221a0a4527 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:59.666076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a123259-2dff-4e59-bf8d-2f221a0a4527 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:59.671204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a123259-2dff-4e59-bf8d-2f221a0a4527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:59.705788 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8f739e8d70a4a69971c92a665fce2cc', 'request_id': 'req-331e8ac5-4cb7-4e31-9876-458b1b7660cf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:59.770651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1a123259-2dff-4e59-bf8d-2f221a0a4527 None None] MFA Rules not processed for user `b80e701a81cb4fa893e55288559010f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:41:59.773472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1a123259-2dff-4e59-bf8d-2f221a0a4527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:59.851232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 870/1739] 10.222.0.45 () {58 vars in 1015 bytes} [Thu Aug 4 07:41:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:41:59.852414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a123259-2dff-4e59-bf8d-2f221a0a4527 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:59.873418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8f739e8d70a4a69971c92a665fce2cc', 'request_id': 'req-331e8ac5-4cb7-4e31-9876-458b1b7660cf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:59.881366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8f739e8d70a4a69971c92a665fce2cc', 'request_id': 'req-331e8ac5-4cb7-4e31-9876-458b1b7660cf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:59.910197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] The token cache is being invalidate because project f8f739e8d70a4a69971c92a665fce2cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:41:59.910528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:59.912483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:41:59.918131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 870/1740] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:41:59 2022] DELETE /identity/v3/projects/f8f739e8d70a4a69971c92a665fce2cc => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:41:59.919301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:41:59.933647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:41:59.960938 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a123259-2dff-4e59-bf8d-2f221a0a4527 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:41:59.974556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525f2f5-791f-4b4b-8c3a-070e41974efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:41:59.975386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f2f5-791f-4b4b-8c3a-070e41974efb admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:41:59.975679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f2f5-791f-4b4b-8c3a-070e41974efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:41:59.975956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f2f5-791f-4b4b-8c3a-070e41974efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:41:59.977202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525f2f5-791f-4b4b-8c3a-070e41974efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:41:59.980570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525f2f5-791f-4b4b-8c3a-070e41974efb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:00.010969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 871/1741] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:00.012387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845be3b1-03c3-49d9-84cd-28ef75c834e6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:00.012678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845be3b1-03c3-49d9-84cd-28ef75c834e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:00.014571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845be3b1-03c3-49d9-84cd-28ef75c834e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:00.067900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:00.070287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-845be3b1-03c3-49d9-84cd-28ef75c834e6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:00.076105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-845be3b1-03c3-49d9-84cd-28ef75c834e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:00.097660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 872/1742] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:00.098567 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845be3b1-03c3-49d9-84cd-28ef75c834e6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:00.099143 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e8ac5-4cb7-4e31-9876-458b1b7660cf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:00.103704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-845be3b1-03c3-49d9-84cd-28ef75c834e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:00.111464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ded702-afc7-4aee-81de-6c36d4349618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:00.112246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ded702-afc7-4aee-81de-6c36d4349618 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:00.112504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ded702-afc7-4aee-81de-6c36d4349618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:00.112679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ded702-afc7-4aee-81de-6c36d4349618 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:00.113392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ded702-afc7-4aee-81de-6c36d4349618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:00.121204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ded702-afc7-4aee-81de-6c36d4349618 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:00.130453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845be3b1-03c3-49d9-84cd-28ef75c834e6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:00.140229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f3dc2-ff30-484c-be5f-697a61e5b356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:00.140875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3dc2-ff30-484c-be5f-697a61e5b356 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:00.141191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3dc2-ff30-484c-be5f-697a61e5b356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:00.141475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3dc2-ff30-484c-be5f-697a61e5b356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:00.147900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-231f3dc2-ff30-484c-be5f-697a61e5b356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:00.188398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f3dc2-ff30-484c-be5f-697a61e5b356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:00.191051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f3dc2-ff30-484c-be5f-697a61e5b356 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:00.225012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 873/1743] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:00.225715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f3dc2-ff30-484c-be5f-697a61e5b356 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:00.239232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-231f3dc2-ff30-484c-be5f-697a61e5b356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:00.273470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f3dc2-ff30-484c-be5f-697a61e5b356 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:00.285949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5bfec-09d6-4314-b839-041d69a86107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:00.286649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5bfec-09d6-4314-b839-041d69a86107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:00.287536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5bfec-09d6-4314-b839-041d69a86107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:00.287536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5bfec-09d6-4314-b839-041d69a86107 admin admin] PATH_INFO: `/v3/projects/85979891703b4e8cb302468f679f0316/users/68119d036cbd415eba57f1774a248a9f/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:00.313817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 871/1744] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:41:59 2022] POST /identity/v3/users => generated 293 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:00.317317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ffade-6703-45fc-87a9-39ae8cb30fcd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:00.317857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ffade-6703-45fc-87a9-39ae8cb30fcd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:00.318413 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ffade-6703-45fc-87a9-39ae8cb30fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:00.332509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5bfec-09d6-4314-b839-041d69a86107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85979891703b4e8cb302468f679f0316, user_id=68119d036cbd415eba57f1774a248a9f, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:00.337701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5bfec-09d6-4314-b839-041d69a86107 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:00.342605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:00.394015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9f9ffade-6703-45fc-87a9-39ae8cb30fcd None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:00.397573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9f9ffade-6703-45fc-87a9-39ae8cb30fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:00.437932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 872/1746] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:00.441408 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 874/1746] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:00 2022] PUT /identity/v3/projects/85979891703b4e8cb302468f679f0316/users/68119d036cbd415eba57f1774a248a9f/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:00.441408 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5bfec-09d6-4314-b839-041d69a86107 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:00.442035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c973c-7ca6-4af5-a880-241615d4b71b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:00.442496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c973c-7ca6-4af5-a880-241615d4b71b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:00.442835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c973c-7ca6-4af5-a880-241615d4b71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:00.486618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d51c973c-7ca6-4af5-a880-241615d4b71b None None] MFA Rules not processed for user `88d78d9c0369431d88ad85c960f6a0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:00.493118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d51c973c-7ca6-4af5-a880-241615d4b71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:00.587404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5bfec-09d6-4314-b839-041d69a86107 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:00.614972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca8522-b9cc-4385-8027-39267ece61cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:00.617289 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca8522-b9cc-4385-8027-39267ece61cc admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:00.618953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca8522-b9cc-4385-8027-39267ece61cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:00.619947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca8522-b9cc-4385-8027-39267ece61cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:00.622884 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca8522-b9cc-4385-8027-39267ece61cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:00.631180 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca8522-b9cc-4385-8027-39267ece61cc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:00.656084 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 873/1747] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:00.656651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 875/1748] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:00.658709 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffc74f-4e29-4ecb-a181-30f00c66041f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:00.659021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c973c-7ca6-4af5-a880-241615d4b71b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:00.659563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffc74f-4e29-4ecb-a181-30f00c66041f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:00.660996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffc74f-4e29-4ecb-a181-30f00c66041f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:00.693870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c973c-7ca6-4af5-a880-241615d4b71b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:00.702434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09ba48-3c5c-402d-8b63-0bad9b761c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:00.703803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09ba48-3c5c-402d-8b63-0bad9b761c04 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:00.706757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09ba48-3c5c-402d-8b63-0bad9b761c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:00.715092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09ba48-3c5c-402d-8b63-0bad9b761c04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:00.718646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09ba48-3c5c-402d-8b63-0bad9b761c04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:00.724200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09ba48-3c5c-402d-8b63-0bad9b761c04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:00.724200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09ba48-3c5c-402d-8b63-0bad9b761c04 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:00.739545 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-47ffc74f-4e29-4ecb-a181-30f00c66041f None None] MFA Rules not processed for user `68119d036cbd415eba57f1774a248a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:00.743107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-47ffc74f-4e29-4ecb-a181-30f00c66041f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:00.774324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 874/1749] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:42:00.787189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09ba48-3c5c-402d-8b63-0bad9b761c04 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:00.812729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09ba48-3c5c-402d-8b63-0bad9b761c04 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:00.830992 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37168da-9190-4b07-b35f-de9df962bd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:00.831720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37168da-9190-4b07-b35f-de9df962bd4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:00.832046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37168da-9190-4b07-b35f-de9df962bd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:00.834299 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37168da-9190-4b07-b35f-de9df962bd4b admin admin] PATH_INFO: `/v3/projects/fc77f15321524aeca9edef3bf7d786b7/users/d40f3cc83f2b4b4ba91d96286a474976/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:00.879132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37168da-9190-4b07-b35f-de9df962bd4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc77f15321524aeca9edef3bf7d786b7, user_id=d40f3cc83f2b4b4ba91d96286a474976, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:00.897870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37168da-9190-4b07-b35f-de9df962bd4b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:00.927680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:00.953341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 876/1750] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:00.955960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffc74f-4e29-4ecb-a181-30f00c66041f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:00.994505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffc74f-4e29-4ecb-a181-30f00c66041f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:00.998827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 875/1751] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:00 2022] PUT /identity/v3/projects/fc77f15321524aeca9edef3bf7d786b7/users/d40f3cc83f2b4b4ba91d96286a474976/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:01.001364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37168da-9190-4b07-b35f-de9df962bd4b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:01.166599 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c78d9-7fef-42b5-a442-473b9ee3d756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:01.167273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c78d9-7fef-42b5-a442-473b9ee3d756 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:01.168246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c78d9-7fef-42b5-a442-473b9ee3d756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:01.168571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c78d9-7fef-42b5-a442-473b9ee3d756 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:01.168755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37168da-9190-4b07-b35f-de9df962bd4b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:01.169871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c78d9-7fef-42b5-a442-473b9ee3d756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:01.172635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c78d9-7fef-42b5-a442-473b9ee3d756 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:01.184013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c373c5-e3b7-47c4-b98a-b479231d62bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:01.185193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c373c5-e3b7-47c4-b98a-b479231d62bd admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:01.185680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c373c5-e3b7-47c4-b98a-b479231d62bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:01.186093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c373c5-e3b7-47c4-b98a-b479231d62bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:01.186931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c373c5-e3b7-47c4-b98a-b479231d62bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:01.192070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c373c5-e3b7-47c4-b98a-b479231d62bd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:01.281237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 877/1752] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:00 2022] POST /identity/v3/projects => generated 363 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:01.286406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c78d9-7fef-42b5-a442-473b9ee3d756 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:01.385074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 876/1753] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:00 2022] POST /identity/v3/projects => generated 357 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:01.385074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c373c5-e3b7-47c4-b98a-b479231d62bd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:01.471517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c373c5-e3b7-47c4-b98a-b479231d62bd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:01.480643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c78d9-7fef-42b5-a442-473b9ee3d756 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:01.487740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe608f-125c-4014-a452-0ce1b08d3940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:01.488437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe608f-125c-4014-a452-0ce1b08d3940 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:01.488718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe608f-125c-4014-a452-0ce1b08d3940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:01.489110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe608f-125c-4014-a452-0ce1b08d3940 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:01.489809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe608f-125c-4014-a452-0ce1b08d3940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:01.491813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe608f-125c-4014-a452-0ce1b08d3940 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:01.507204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c3ee5-2bd9-468e-b650-56a6c4902f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:01.509333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c3ee5-2bd9-468e-b650-56a6c4902f98 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:01.515066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c3ee5-2bd9-468e-b650-56a6c4902f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:01.515066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c3ee5-2bd9-468e-b650-56a6c4902f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:01.515066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c3ee5-2bd9-468e-b650-56a6c4902f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:01.515066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c3ee5-2bd9-468e-b650-56a6c4902f98 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:01.536469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 878/1754] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:01.536469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c3ee5-2bd9-468e-b650-56a6c4902f98 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:01.546588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c3ee5-2bd9-468e-b650-56a6c4902f98 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:01.557178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59276e8-1e4b-4c7a-a1bb-e85c2967c61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:01.557178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59276e8-1e4b-4c7a-a1bb-e85c2967c61a admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:01.557178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59276e8-1e4b-4c7a-a1bb-e85c2967c61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:01.557178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59276e8-1e4b-4c7a-a1bb-e85c2967c61a admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:01.557644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59276e8-1e4b-4c7a-a1bb-e85c2967c61a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:01.558724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59276e8-1e4b-4c7a-a1bb-e85c2967c61a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:01.596221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 877/1755] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:01 2022] POST /identity/v3/users => generated 282 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:01.597068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe608f-125c-4014-a452-0ce1b08d3940 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:01.618316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:01.622576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe608f-125c-4014-a452-0ce1b08d3940 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:01.636054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26650237-93ca-4334-a775-c0e390c6e636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:01.636937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650237-93ca-4334-a775-c0e390c6e636 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:01.637054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650237-93ca-4334-a775-c0e390c6e636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:01.637247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650237-93ca-4334-a775-c0e390c6e636 admin admin] PATH_INFO: `/v3/projects/fc77f15321524aeca9edef3bf7d786b7/users/d40f3cc83f2b4b4ba91d96286a474976/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:01.642305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26650237-93ca-4334-a775-c0e390c6e636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc77f15321524aeca9edef3bf7d786b7, user_id=d40f3cc83f2b4b4ba91d96286a474976, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:01.645224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26650237-93ca-4334-a775-c0e390c6e636 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:01.691216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 878/1756] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:01 2022] PUT /identity/v3/projects/fc77f15321524aeca9edef3bf7d786b7/users/d40f3cc83f2b4b4ba91d96286a474976/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:01.692152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26650237-93ca-4334-a775-c0e390c6e636 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:01.697167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 879/1757] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:01 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:01.704801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59276e8-1e4b-4c7a-a1bb-e85c2967c61a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:01.714676 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:01.734046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59276e8-1e4b-4c7a-a1bb-e85c2967c61a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:01.838629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26650237-93ca-4334-a775-c0e390c6e636 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:01.856424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f71f3-3f29-4f88-ae74-6847d2c6d318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:01.856424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f71f3-3f29-4f88-ae74-6847d2c6d318 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:01.856424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f71f3-3f29-4f88-ae74-6847d2c6d318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:01.856424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f71f3-3f29-4f88-ae74-6847d2c6d318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:01.856947 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f71f3-3f29-4f88-ae74-6847d2c6d318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:01.858161 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f71f3-3f29-4f88-ae74-6847d2c6d318 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:01.893422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 879/1758] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:01.894854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df703fc-d6c5-4eaf-9adb-7f85eb6f5d4d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:01.895144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df703fc-d6c5-4eaf-9adb-7f85eb6f5d4d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:01.895424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df703fc-d6c5-4eaf-9adb-7f85eb6f5d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:01.900753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceeb730-19eb-4c5d-a25c-337505430e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:01.901405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceeb730-19eb-4c5d-a25c-337505430e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:01.901770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceeb730-19eb-4c5d-a25c-337505430e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:01.906415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceeb730-19eb-4c5d-a25c-337505430e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:01.909532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceeb730-19eb-4c5d-a25c-337505430e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:01.912936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceeb730-19eb-4c5d-a25c-337505430e2d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:01.970712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3df703fc-d6c5-4eaf-9adb-7f85eb6f5d4d None None] MFA Rules not processed for user `d40f3cc83f2b4b4ba91d96286a474976`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:01.972687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 880/1759] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:01.973440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3df703fc-d6c5-4eaf-9adb-7f85eb6f5d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:01.977081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceeb730-19eb-4c5d-a25c-337505430e2d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:01.994930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceeb730-19eb-4c5d-a25c-337505430e2d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:02.004219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e463430-8db3-4733-9eda-50473e6b08a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:02.004932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e463430-8db3-4733-9eda-50473e6b08a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:02.005244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e463430-8db3-4733-9eda-50473e6b08a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:02.005528 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e463430-8db3-4733-9eda-50473e6b08a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:02.006784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e463430-8db3-4733-9eda-50473e6b08a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:02.011616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e463430-8db3-4733-9eda-50473e6b08a6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:02.031625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 881/1760] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:02.032631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e463430-8db3-4733-9eda-50473e6b08a6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:02.043462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e463430-8db3-4733-9eda-50473e6b08a6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:02.051804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26823b8-ce68-4819-aabc-faaf5fa90746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:02.051804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26823b8-ce68-4819-aabc-faaf5fa90746 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:02.051804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26823b8-ce68-4819-aabc-faaf5fa90746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:02.052208 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26823b8-ce68-4819-aabc-faaf5fa90746 admin admin] PATH_INFO: `/v3/projects/5918d40605d84f1fab05e39ae4a59d9c/users/8c6e0d58d5e54fc18b0dc7f88bd4b5c9/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:02.073257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26823b8-ce68-4819-aabc-faaf5fa90746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5918d40605d84f1fab05e39ae4a59d9c, user_id=8c6e0d58d5e54fc18b0dc7f88bd4b5c9, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:02.076478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26823b8-ce68-4819-aabc-faaf5fa90746 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:02.156281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 880/1761] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:02.158731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df703fc-d6c5-4eaf-9adb-7f85eb6f5d4d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:02.173651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 882/1762] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:02 2022] PUT /identity/v3/projects/5918d40605d84f1fab05e39ae4a59d9c/users/8c6e0d58d5e54fc18b0dc7f88bd4b5c9/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:02.181674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26823b8-ce68-4819-aabc-faaf5fa90746 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:02.181776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df703fc-d6c5-4eaf-9adb-7f85eb6f5d4d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:02.181860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c26823b8-ce68-4819-aabc-faaf5fa90746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:02.314043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88dd8c-bdc0-47fe-9b28-a80bcef6028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:02.316583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88dd8c-bdc0-47fe-9b28-a80bcef6028f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:02.316942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88dd8c-bdc0-47fe-9b28-a80bcef6028f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:02.317442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88dd8c-bdc0-47fe-9b28-a80bcef6028f admin admin] PATH_INFO: `/v3/projects/96c4257a34cf4ffd89124cfe7511e56f/users/325a884b94fb4d82b5f9a7335aee1173/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:02.332886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26823b8-ce68-4819-aabc-faaf5fa90746 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:02.340955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40193e6-9a30-4a1b-95df-5e4fa0c4fd2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:02.342019 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40193e6-9a30-4a1b-95df-5e4fa0c4fd2f service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:02.342136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40193e6-9a30-4a1b-95df-5e4fa0c4fd2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:02.343422 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40193e6-9a30-4a1b-95df-5e4fa0c4fd2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:02.351786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88dd8c-bdc0-47fe-9b28-a80bcef6028f admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c4257a34cf4ffd89124cfe7511e56f, user_id=325a884b94fb4d82b5f9a7335aee1173, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:02.358971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88dd8c-bdc0-47fe-9b28-a80bcef6028f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:02.436357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:02.512030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 881/1763] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:02 2022] PUT /identity/v3/projects/96c4257a34cf4ffd89124cfe7511e56f/users/325a884b94fb4d82b5f9a7335aee1173/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:02.513242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88dd8c-bdc0-47fe-9b28-a80bcef6028f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:02.517703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40193e6-9a30-4a1b-95df-5e4fa0c4fd2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:02.519789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40193e6-9a30-4a1b-95df-5e4fa0c4fd2f service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:02.550985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 883/1764] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:02.553950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eb5e2-ef17-44ce-9863-a2a00b84ab7a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:02.555207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eb5e2-ef17-44ce-9863-a2a00b84ab7a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:02.555622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eb5e2-ef17-44ce-9863-a2a00b84ab7a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:02.558657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 884/1765] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:42:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:42:02.572732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad6c73-9a9e-4057-8a4d-94b8535be17d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:02.572732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad6c73-9a9e-4057-8a4d-94b8535be17d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:02.572732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad6c73-9a9e-4057-8a4d-94b8535be17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:02.627917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:02.665935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7aad6c73-9a9e-4057-8a4d-94b8535be17d None None] MFA Rules not processed for user `325a884b94fb4d82b5f9a7335aee1173`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:02.665935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7aad6c73-9a9e-4057-8a4d-94b8535be17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:02.787939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88dd8c-bdc0-47fe-9b28-a80bcef6028f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:02.800317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b71df7-28f5-4b95-a31a-d255f5fe14a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:02.800962 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b71df7-28f5-4b95-a31a-d255f5fe14a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:02.801298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b71df7-28f5-4b95-a31a-d255f5fe14a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:02.801583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b71df7-28f5-4b95-a31a-d255f5fe14a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:02.802662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b71df7-28f5-4b95-a31a-d255f5fe14a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:02.804917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b71df7-28f5-4b95-a31a-d255f5fe14a7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:02.810002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 885/1766] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:02.820366 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81119c20-e2cd-4ff4-9d00-f822174142a0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:02.822029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81119c20-e2cd-4ff4-9d00-f822174142a0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:02.823337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81119c20-e2cd-4ff4-9d00-f822174142a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:02.831784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 882/1767] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:02.835604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b71df7-28f5-4b95-a31a-d255f5fe14a7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:02.877462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-81119c20-e2cd-4ff4-9d00-f822174142a0 None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:02.879503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-81119c20-e2cd-4ff4-9d00-f822174142a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:02.888112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b71df7-28f5-4b95-a31a-d255f5fe14a7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:02.911217 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cbccc-5d1f-4a36-b96e-646aa6a672a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:02.911217 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cbccc-5d1f-4a36-b96e-646aa6a672a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:02.911217 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cbccc-5d1f-4a36-b96e-646aa6a672a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:02.911217 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cbccc-5d1f-4a36-b96e-646aa6a672a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:02.911658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cbccc-5d1f-4a36-b96e-646aa6a672a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:02.928461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cbccc-5d1f-4a36-b96e-646aa6a672a5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:03.149098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 883/1768] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:02 2022] POST /identity/v3/projects => generated 365 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:03.151949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cbccc-5d1f-4a36-b96e-646aa6a672a5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:03.161208 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 886/1769] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:42:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:03.173403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81119c20-e2cd-4ff4-9d00-f822174142a0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:03.278398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81119c20-e2cd-4ff4-9d00-f822174142a0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:03.415237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cbccc-5d1f-4a36-b96e-646aa6a672a5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:03.428643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54f6e4-398e-4013-877d-f0bb54e60172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:03.429850 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54f6e4-398e-4013-877d-f0bb54e60172 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:03.430083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54f6e4-398e-4013-877d-f0bb54e60172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:03.430624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54f6e4-398e-4013-877d-f0bb54e60172 admin admin] PATH_INFO: `/v3/projects/5918d40605d84f1fab05e39ae4a59d9c/users/8c6e0d58d5e54fc18b0dc7f88bd4b5c9/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:03.437992 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54f6e4-398e-4013-877d-f0bb54e60172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5918d40605d84f1fab05e39ae4a59d9c, user_id=8c6e0d58d5e54fc18b0dc7f88bd4b5c9, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:03.445479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54f6e4-398e-4013-877d-f0bb54e60172 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:03.475351 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 884/1770] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:03 2022] PUT /identity/v3/projects/5918d40605d84f1fab05e39ae4a59d9c/users/8c6e0d58d5e54fc18b0dc7f88bd4b5c9/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:03.490843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31144c13-4ad2-45c5-95e8-36697964db8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:03.491406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31144c13-4ad2-45c5-95e8-36697964db8e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:03.491654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31144c13-4ad2-45c5-95e8-36697964db8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:03.491899 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80b685-ad54-4943-a61f-edeff20a6364 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:03.492043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31144c13-4ad2-45c5-95e8-36697964db8e admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:03.492225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80b685-ad54-4943-a61f-edeff20a6364 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:03.492662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31144c13-4ad2-45c5-95e8-36697964db8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:03.494597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31144c13-4ad2-45c5-95e8-36697964db8e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:03.494717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80b685-ad54-4943-a61f-edeff20a6364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:03.544560 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5b80b685-ad54-4943-a61f-edeff20a6364 None None] MFA Rules not processed for user `8c6e0d58d5e54fc18b0dc7f88bd4b5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:03.545820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5b80b685-ad54-4943-a61f-edeff20a6364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:03.604083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 887/1771] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:03 2022] POST /identity/v3/users => generated 289 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:03.608853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31144c13-4ad2-45c5-95e8-36697964db8e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:03.644069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31144c13-4ad2-45c5-95e8-36697964db8e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:03.790629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 885/1772] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:03.796291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:03.821495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80b685-ad54-4943-a61f-edeff20a6364 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:03.824653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5b80b685-ad54-4943-a61f-edeff20a6364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:03.831437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f8136-6e65-47d8-b4a1-c125efd07836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:03.832908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f8136-6e65-47d8-b4a1-c125efd07836 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:03.833506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f8136-6e65-47d8-b4a1-c125efd07836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:03.833793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f8136-6e65-47d8-b4a1-c125efd07836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:03.838212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f8136-6e65-47d8-b4a1-c125efd07836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:03.839938 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f8136-6e65-47d8-b4a1-c125efd07836 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:03.876989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 888/1773] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:03.881071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f8136-6e65-47d8-b4a1-c125efd07836 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:03.928092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f8136-6e65-47d8-b4a1-c125efd07836 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:03.946933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77c15d-f209-491a-b075-168848c53a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:03.947684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77c15d-f209-491a-b075-168848c53a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:03.948026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77c15d-f209-491a-b075-168848c53a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:03.948303 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77c15d-f209-491a-b075-168848c53a00 admin admin] PATH_INFO: `/v3/projects/d4dcdbb3710f4c31816fdf842d008e8e/users/8786ad461bca4b03a1bdff82e0605db4/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:03.984176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77c15d-f209-491a-b075-168848c53a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4dcdbb3710f4c31816fdf842d008e8e, user_id=8786ad461bca4b03a1bdff82e0605db4, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:03.987879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77c15d-f209-491a-b075-168848c53a00 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:04.021359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:04.061535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80b685-ad54-4943-a61f-edeff20a6364 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:04.073932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 889/1774] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:03 2022] PUT /identity/v3/projects/d4dcdbb3710f4c31816fdf842d008e8e/users/8786ad461bca4b03a1bdff82e0605db4/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:04.075174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77c15d-f209-491a-b075-168848c53a00 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:04.246701 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad1893-bce9-4152-baca-8f240fcd138d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:04.247481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad1893-bce9-4152-baca-8f240fcd138d service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:04.247866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad1893-bce9-4152-baca-8f240fcd138d service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:04.248272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad1893-bce9-4152-baca-8f240fcd138d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:04.302492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77c15d-f209-491a-b075-168848c53a00 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:04.317950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376800ad-8d4c-4814-81c4-0ad701f7236c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:04.319490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376800ad-8d4c-4814-81c4-0ad701f7236c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:04.319892 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376800ad-8d4c-4814-81c4-0ad701f7236c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:04.320302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376800ad-8d4c-4814-81c4-0ad701f7236c admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:04.321183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376800ad-8d4c-4814-81c4-0ad701f7236c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:04.323051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376800ad-8d4c-4814-81c4-0ad701f7236c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:04.372113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 890/1775] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:04.374536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376800ad-8d4c-4814-81c4-0ad701f7236c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:04.414665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376800ad-8d4c-4814-81c4-0ad701f7236c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:04.433711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8eff97-9644-4bba-99ff-b4576bb06347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:04.434518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eff97-9644-4bba-99ff-b4576bb06347 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:04.435084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eff97-9644-4bba-99ff-b4576bb06347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:04.436313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eff97-9644-4bba-99ff-b4576bb06347 admin admin] PATH_INFO: `/v3/projects/d4dcdbb3710f4c31816fdf842d008e8e/users/8786ad461bca4b03a1bdff82e0605db4/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:04.446920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8eff97-9644-4bba-99ff-b4576bb06347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4dcdbb3710f4c31816fdf842d008e8e, user_id=8786ad461bca4b03a1bdff82e0605db4, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:04.459442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad1893-bce9-4152-baca-8f240fcd138d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:04.461926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad1893-bce9-4152-baca-8f240fcd138d service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:04.463641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8eff97-9644-4bba-99ff-b4576bb06347 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:04.493024 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 891/1776] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:04 2022] PUT /identity/v3/projects/d4dcdbb3710f4c31816fdf842d008e8e/users/8786ad461bca4b03a1bdff82e0605db4/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:04.495333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df401f-d68b-4721-85fd-72d0078d8a6a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:04.496762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df401f-d68b-4721-85fd-72d0078d8a6a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:04.498336 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df401f-d68b-4721-85fd-72d0078d8a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:04.505056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 886/1777] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:04.557613 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-14df401f-d68b-4721-85fd-72d0078d8a6a None None] MFA Rules not processed for user `8786ad461bca4b03a1bdff82e0605db4`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:04.559554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-14df401f-d68b-4721-85fd-72d0078d8a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:04.620682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad1893-bce9-4152-baca-8f240fcd138d service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:04.649419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad1893-bce9-4152-baca-8f240fcd138d service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:04.793188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:04.825477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 892/1778] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:04.833463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df401f-d68b-4721-85fd-72d0078d8a6a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:04.928779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f49355-812f-49f3-a67b-5f807e20c431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:04.929456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f49355-812f-49f3-a67b-5f807e20c431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:04.929745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f49355-812f-49f3-a67b-5f807e20c431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:04.930026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f49355-812f-49f3-a67b-5f807e20c431 admin admin] PATH_INFO: `/v3/users/ba98b5d8e4b84af99668d8ffe7e9d047` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:04.933001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f49355-812f-49f3-a67b-5f807e20c431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba98b5d8e4b84af99668d8ffe7e9d047)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:04.935376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f49355-812f-49f3-a67b-5f807e20c431 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:05.020014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:05.040686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df401f-d68b-4721-85fd-72d0078d8a6a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:05.052417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3552cc3-a2ba-4338-87a0-9a36a9ed5a97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:05.053573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3552cc3-a2ba-4338-87a0-9a36a9ed5a97 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:05.054156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3552cc3-a2ba-4338-87a0-9a36a9ed5a97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:05.054685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3552cc3-a2ba-4338-87a0-9a36a9ed5a97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:05.188587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3552cc3-a2ba-4338-87a0-9a36a9ed5a97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:05.191331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3552cc3-a2ba-4338-87a0-9a36a9ed5a97 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:05.209837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 893/1779] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:05.210856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c8f49355-812f-49f3-a67b-5f807e20c431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba98b5d8e4b84af99668d8ffe7e9d047'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:05.211882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c8f49355-812f-49f3-a67b-5f807e20c431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba98b5d8e4b84af99668d8ffe7e9d047'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:05.232303 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3552cc3-a2ba-4338-87a0-9a36a9ed5a97 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:05.253299 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c8f49355-812f-49f3-a67b-5f807e20c431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba98b5d8e4b84af99668d8ffe7e9d047'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:05.255656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3552cc3-a2ba-4338-87a0-9a36a9ed5a97 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:05.261913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c8f4-b2d1-42e9-a910-3ea31a28dcef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:05.264386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c8f4-b2d1-42e9-a910-3ea31a28dcef service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:05.264386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c8f4-b2d1-42e9-a910-3ea31a28dcef service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:05.264386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c8f4-b2d1-42e9-a910-3ea31a28dcef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:05.268185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7170c8f4-b2d1-42e9-a910-3ea31a28dcef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:05.284993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170c8f4-b2d1-42e9-a910-3ea31a28dcef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:05.287960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170c8f4-b2d1-42e9-a910-3ea31a28dcef service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:05.295510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c8f49355-812f-49f3-a67b-5f807e20c431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba98b5d8e4b84af99668d8ffe7e9d047'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:05.321753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 894/1780] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:05.411928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c8f49355-812f-49f3-a67b-5f807e20c431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba98b5d8e4b84af99668d8ffe7e9d047'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:05.420682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 887/1781] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:04 2022] DELETE /identity/v3/users/ba98b5d8e4b84af99668d8ffe7e9d047 => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:05.426265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391b06f-f797-486f-8753-43d83c711809 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:05.426650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391b06f-f797-486f-8753-43d83c711809 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:05.427045 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391b06f-f797-486f-8753-43d83c711809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:05.454329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8f9b8-9653-46a6-a112-ef708f10962e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:05.454617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8f9b8-9653-46a6-a112-ef708f10962e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:05.455078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8f9b8-9653-46a6-a112-ef708f10962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:05.468066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5391b06f-f797-486f-8753-43d83c711809 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:05.469874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5391b06f-f797-486f-8753-43d83c711809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:05.484340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fdd8f9b8-9653-46a6-a112-ef708f10962e None None] MFA Rules not processed for user `5509519254574e50a70c84ae895acd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:05.486760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fdd8f9b8-9653-46a6-a112-ef708f10962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:05.563785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 888/1782] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:05.576072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 895/1783] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:05.583457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8f9b8-9653-46a6-a112-ef708f10962e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:05.584622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fdd8f9b8-9653-46a6-a112-ef708f10962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:05.663890 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8f9b8-9653-46a6-a112-ef708f10962e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:05.668021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1aad6-0d6f-4e53-83ea-34e90c784b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:05.668468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1aad6-0d6f-4e53-83ea-34e90c784b0b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:05.668755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1aad6-0d6f-4e53-83ea-34e90c784b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:05.669031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1aad6-0d6f-4e53-83ea-34e90c784b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:05.680228 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1aad6-0d6f-4e53-83ea-34e90c784b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:05.681688 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1aad6-0d6f-4e53-83ea-34e90c784b0b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:05.685876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:05.692886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 889/1784] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:05.721802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391b06f-f797-486f-8753-43d83c711809 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:05.726672 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5391b06f-f797-486f-8753-43d83c711809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:05.734367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:05.850625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391b06f-f797-486f-8753-43d83c711809 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:05.856496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c5397-8fe8-4d75-8d5b-1353b8d553ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:05.858007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c5397-8fe8-4d75-8d5b-1353b8d553ef service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:05.859446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c5397-8fe8-4d75-8d5b-1353b8d553ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:05.860426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c5397-8fe8-4d75-8d5b-1353b8d553ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:05.862810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4e4c5397-8fe8-4d75-8d5b-1353b8d553ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:05.972917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c5397-8fe8-4d75-8d5b-1353b8d553ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:05.974887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c5397-8fe8-4d75-8d5b-1353b8d553ef service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:05.988494 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 896/1785] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:06.658501 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1aad6-0d6f-4e53-83ea-34e90c784b0b service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:06.660182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0ce1aad6-0d6f-4e53-83ea-34e90c784b0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:06.679329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1aad6-0d6f-4e53-83ea-34e90c784b0b service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:06.691354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:06.692089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:06.692449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:06.702984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] PATH_INFO: `/v3/projects/1c7bb7290e8e444faa0e82556b45d0d9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:06.702984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7bb7290e8e444faa0e82556b45d0d9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:06.702984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:06.854895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7bb7290e8e444faa0e82556b45d0d9', 'request_id': 'req-58129386-66bc-495a-b166-48a621fef6db'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:06.937812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7bb7290e8e444faa0e82556b45d0d9', 'request_id': 'req-58129386-66bc-495a-b166-48a621fef6db'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:06.946270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7bb7290e8e444faa0e82556b45d0d9', 'request_id': 'req-58129386-66bc-495a-b166-48a621fef6db'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:06.980297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] The token cache is being invalidate because project 1c7bb7290e8e444faa0e82556b45d0d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:06.980598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:06.984416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:06.989840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 890/1786] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:06 2022] DELETE /identity/v3/projects/1c7bb7290e8e444faa0e82556b45d0d9 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:06.990893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c5397-8fe8-4d75-8d5b-1353b8d553ef service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:07.007503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c5397-8fe8-4d75-8d5b-1353b8d553ef service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:07.103812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f53ff-aaee-4138-9a8b-e71ef288d868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:07.104512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f53ff-aaee-4138-9a8b-e71ef288d868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:07.104806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f53ff-aaee-4138-9a8b-e71ef288d868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:07.105076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f53ff-aaee-4138-9a8b-e71ef288d868 admin admin] PATH_INFO: `/v3/users/d40f3cc83f2b4b4ba91d96286a474976` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:07.116822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f53ff-aaee-4138-9a8b-e71ef288d868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d40f3cc83f2b4b4ba91d96286a474976)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:07.118812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f53ff-aaee-4138-9a8b-e71ef288d868 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:07.346479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fc9f53ff-aaee-4138-9a8b-e71ef288d868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd40f3cc83f2b4b4ba91d96286a474976'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:07.349152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fc9f53ff-aaee-4138-9a8b-e71ef288d868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd40f3cc83f2b4b4ba91d96286a474976'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:07.349788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fc9f53ff-aaee-4138-9a8b-e71ef288d868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd40f3cc83f2b4b4ba91d96286a474976'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:07.362859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:07.366205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fc9f53ff-aaee-4138-9a8b-e71ef288d868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd40f3cc83f2b4b4ba91d96286a474976'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:07.474961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fc9f53ff-aaee-4138-9a8b-e71ef288d868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd40f3cc83f2b4b4ba91d96286a474976'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:07.491122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 897/1787] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:06 2022] DELETE /identity/v3/users/d40f3cc83f2b4b4ba91d96286a474976 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:08.218282 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:08.220258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:08.261396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58129386-66bc-495a-b166-48a621fef6db admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:08.329347 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:08.330391 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:08.330830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:08.331056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] PATH_INFO: `/v3/projects/fc77f15321524aeca9edef3bf7d786b7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:08.332366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc77f15321524aeca9edef3bf7d786b7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:08.334548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:08.487845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc77f15321524aeca9edef3bf7d786b7', 'request_id': 'req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:08.565667 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc77f15321524aeca9edef3bf7d786b7', 'request_id': 'req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:08.574379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc77f15321524aeca9edef3bf7d786b7', 'request_id': 'req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:08.628396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] The token cache is being invalidate because project fc77f15321524aeca9edef3bf7d786b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:08.632051 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:08.635186 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c24e94f5-7cd5-4bd7-96fa-ae953aaf61ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:08.638030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 891/1788] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:08 2022] DELETE /identity/v3/projects/fc77f15321524aeca9edef3bf7d786b7 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:08.643353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bdadd-a3ba-47c0-abd9-482bde08a48e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:08.644229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bdadd-a3ba-47c0-abd9-482bde08a48e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:08.644229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bdadd-a3ba-47c0-abd9-482bde08a48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:08.666062 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a68bdadd-a3ba-47c0-abd9-482bde08a48e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:08.667428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a68bdadd-a3ba-47c0-abd9-482bde08a48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:08.832325 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 898/1789] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:08.841438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb5b3f-3a2e-475e-87dc-79d496d07c4a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:08.842093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb5b3f-3a2e-475e-87dc-79d496d07c4a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:08.842598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb5b3f-3a2e-475e-87dc-79d496d07c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:08.868544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f4cb5b3f-3a2e-475e-87dc-79d496d07c4a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:08.870406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f4cb5b3f-3a2e-475e-87dc-79d496d07c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:08.884454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 892/1790] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:08.888979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bdadd-a3ba-47c0-abd9-482bde08a48e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:08.905872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bdadd-a3ba-47c0-abd9-482bde08a48e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:08.912353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585cecf-f67f-42b3-925e-be635d016021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:08.914283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585cecf-f67f-42b3-925e-be635d016021 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:08.914283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585cecf-f67f-42b3-925e-be635d016021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:08.914283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585cecf-f67f-42b3-925e-be635d016021 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:08.914914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585cecf-f67f-42b3-925e-be635d016021 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:08.915231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585cecf-f67f-42b3-925e-be635d016021 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:08.916944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585cecf-f67f-42b3-925e-be635d016021 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:08.932708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 899/1791] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:42:08.937435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb5b3f-3a2e-475e-87dc-79d496d07c4a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:08.943814 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:08.948186 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb5b3f-3a2e-475e-87dc-79d496d07c4a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:08.958855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e98c4-348e-48de-a189-e6b6e121d35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:08.960309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e98c4-348e-48de-a189-e6b6e121d35e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:08.966040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e98c4-348e-48de-a189-e6b6e121d35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:08.966040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e98c4-348e-48de-a189-e6b6e121d35e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:08.966040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e98c4-348e-48de-a189-e6b6e121d35e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:08.966040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e98c4-348e-48de-a189-e6b6e121d35e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:09.102059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 893/1792] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:08 2022] POST /identity/v3/projects => generated 363 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:09.106765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585cecf-f67f-42b3-925e-be635d016021 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:09.122034 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585cecf-f67f-42b3-925e-be635d016021 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:09.263785 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433497c9-728f-4a20-b3f8-a0c2064df6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:09.264998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433497c9-728f-4a20-b3f8-a0c2064df6c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:09.265501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433497c9-728f-4a20-b3f8-a0c2064df6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:09.266033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433497c9-728f-4a20-b3f8-a0c2064df6c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:09.266945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433497c9-728f-4a20-b3f8-a0c2064df6c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:09.269586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433497c9-728f-4a20-b3f8-a0c2064df6c3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:09.371207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 900/1793] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:09 2022] POST /identity/v3/users => generated 282 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:09.375757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e98c4-348e-48de-a189-e6b6e121d35e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:09.394805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e98c4-348e-48de-a189-e6b6e121d35e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:09.409701 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628ba34-ee63-412c-ad4b-8451403c6b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:09.410500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628ba34-ee63-412c-ad4b-8451403c6b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:09.410944 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628ba34-ee63-412c-ad4b-8451403c6b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:09.411140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628ba34-ee63-412c-ad4b-8451403c6b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:09.414749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628ba34-ee63-412c-ad4b-8451403c6b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:09.416414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628ba34-ee63-412c-ad4b-8451403c6b1b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:09.445984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 894/1794] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:09.447213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433497c9-728f-4a20-b3f8-a0c2064df6c3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:09.461704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433497c9-728f-4a20-b3f8-a0c2064df6c3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:09.472859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941b53c-f3c3-44c5-b8e2-18be26b9175d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:09.473562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b53c-f3c3-44c5-b8e2-18be26b9175d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:09.473787 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b53c-f3c3-44c5-b8e2-18be26b9175d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:09.474222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b53c-f3c3-44c5-b8e2-18be26b9175d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:09.474942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941b53c-f3c3-44c5-b8e2-18be26b9175d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:09.476682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941b53c-f3c3-44c5-b8e2-18be26b9175d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:09.492645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 901/1795] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:09.493361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628ba34-ee63-412c-ad4b-8451403c6b1b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:09.505577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628ba34-ee63-412c-ad4b-8451403c6b1b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:09.511078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a48467-488c-40a8-9771-1111c07dcbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:09.516305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a48467-488c-40a8-9771-1111c07dcbd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:09.516305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a48467-488c-40a8-9771-1111c07dcbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:09.516305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a48467-488c-40a8-9771-1111c07dcbd6 admin admin] PATH_INFO: `/v3/projects/be2daa8d0ac74d9a9a67f77d367da9db/users/718253b715c245c5907acb407d5112e4/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:09.533085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a48467-488c-40a8-9771-1111c07dcbd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2daa8d0ac74d9a9a67f77d367da9db, user_id=718253b715c245c5907acb407d5112e4, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:09.536576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a48467-488c-40a8-9771-1111c07dcbd6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:09.666707 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 895/1796] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:09 2022] PUT /identity/v3/projects/be2daa8d0ac74d9a9a67f77d367da9db/users/718253b715c245c5907acb407d5112e4/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:09.672798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3256b53-8dfe-447d-8f5c-15e90b38e6fd None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:09.673415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3256b53-8dfe-447d-8f5c-15e90b38e6fd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:09.673662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3256b53-8dfe-447d-8f5c-15e90b38e6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:09.706720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a3256b53-8dfe-447d-8f5c-15e90b38e6fd None None] MFA Rules not processed for user `718253b715c245c5907acb407d5112e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:09.708285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a3256b53-8dfe-447d-8f5c-15e90b38e6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:09.758076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:09.820173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 902/1797] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:09.823086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a48467-488c-40a8-9771-1111c07dcbd6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:09.919504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a48467-488c-40a8-9771-1111c07dcbd6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:09.941067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b31d52-1cca-491a-b35c-8c8f874c5e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:09.941777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b31d52-1cca-491a-b35c-8c8f874c5e53 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:09.942067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b31d52-1cca-491a-b35c-8c8f874c5e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:09.942348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b31d52-1cca-491a-b35c-8c8f874c5e53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:09.943042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b31d52-1cca-491a-b35c-8c8f874c5e53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:09.945055 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b31d52-1cca-491a-b35c-8c8f874c5e53 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:10.160327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3256b53-8dfe-447d-8f5c-15e90b38e6fd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:10.161913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a3256b53-8dfe-447d-8f5c-15e90b38e6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:10.178751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3256b53-8dfe-447d-8f5c-15e90b38e6fd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:10.183351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982e82f-e2ca-47e4-b9ce-f4fefae03111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:10.184104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982e82f-e2ca-47e4-b9ce-f4fefae03111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:10.184399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982e82f-e2ca-47e4-b9ce-f4fefae03111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:10.184681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982e82f-e2ca-47e4-b9ce-f4fefae03111 admin admin] PATH_INFO: `/v3/users/325a884b94fb4d82b5f9a7335aee1173` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:10.185893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982e82f-e2ca-47e4-b9ce-f4fefae03111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=325a884b94fb4d82b5f9a7335aee1173)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:10.188473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982e82f-e2ca-47e4-b9ce-f4fefae03111 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:10.265105 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 896/1798] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:09 2022] POST /identity/v3/projects => generated 365 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:10.269087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b31d52-1cca-491a-b35c-8c8f874c5e53 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:10.346775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b31d52-1cca-491a-b35c-8c8f874c5e53 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:10.355308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc78ba-a949-4fc5-bded-02c878854cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:10.355308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc78ba-a949-4fc5-bded-02c878854cee admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:10.355308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc78ba-a949-4fc5-bded-02c878854cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:10.355308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc78ba-a949-4fc5-bded-02c878854cee admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:10.355727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc78ba-a949-4fc5-bded-02c878854cee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:10.356915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc78ba-a949-4fc5-bded-02c878854cee admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:11.450338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 897/1799] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:10 2022] POST /identity/v3/users => generated 289 bytes in 1178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:11.450338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc78ba-a949-4fc5-bded-02c878854cee admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:11.461249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc78ba-a949-4fc5-bded-02c878854cee admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:11.465965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bc097-220c-49db-842e-f929bbc737ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:11.466637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bc097-220c-49db-842e-f929bbc737ca admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:11.467404 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bc097-220c-49db-842e-f929bbc737ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:11.467619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bc097-220c-49db-842e-f929bbc737ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:11.468401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bc097-220c-49db-842e-f929bbc737ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:11.470424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bc097-220c-49db-842e-f929bbc737ca admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:11.481522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 898/1800] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:11.485811 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bc097-220c-49db-842e-f929bbc737ca admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:11.503825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bc097-220c-49db-842e-f929bbc737ca admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:11.510463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b865a-1932-41cc-b0d8-c8ef5014e3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:11.511159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b865a-1932-41cc-b0d8-c8ef5014e3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:11.511602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b865a-1932-41cc-b0d8-c8ef5014e3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:11.511886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b865a-1932-41cc-b0d8-c8ef5014e3ed admin admin] PATH_INFO: `/v3/projects/60c6387c01704120a9aa52f38deffb87/users/7df01e838c834005ac869461375d2c9a/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:11.523093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:11.540564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b865a-1932-41cc-b0d8-c8ef5014e3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c6387c01704120a9aa52f38deffb87, user_id=7df01e838c834005ac869461375d2c9a, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:11.540734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1982e82f-e2ca-47e4-b9ce-f4fefae03111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325a884b94fb4d82b5f9a7335aee1173'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:11.541294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1982e82f-e2ca-47e4-b9ce-f4fefae03111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325a884b94fb4d82b5f9a7335aee1173'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:11.541942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1982e82f-e2ca-47e4-b9ce-f4fefae03111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325a884b94fb4d82b5f9a7335aee1173'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:11.543597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b865a-1932-41cc-b0d8-c8ef5014e3ed admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:11.556318 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1982e82f-e2ca-47e4-b9ce-f4fefae03111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325a884b94fb4d82b5f9a7335aee1173'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:11.601035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 899/1801] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:11 2022] PUT /identity/v3/projects/60c6387c01704120a9aa52f38deffb87/users/7df01e838c834005ac869461375d2c9a/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:11.604603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b865a-1932-41cc-b0d8-c8ef5014e3ed admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:11.606496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f08b865a-1932-41cc-b0d8-c8ef5014e3ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:11.640506 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1982e82f-e2ca-47e4-b9ce-f4fefae03111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325a884b94fb4d82b5f9a7335aee1173'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:11.670078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 903/1802] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:10 2022] DELETE /identity/v3/users/325a884b94fb4d82b5f9a7335aee1173 => generated 0 bytes in 1510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:11.675292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab02a7-40ff-41bb-8ae4-f4012f5fce29 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:11.675751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab02a7-40ff-41bb-8ae4-f4012f5fce29 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:11.676236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab02a7-40ff-41bb-8ae4-f4012f5fce29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:11.689310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b865a-1932-41cc-b0d8-c8ef5014e3ed admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:11.698804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-51ab02a7-40ff-41bb-8ae4-f4012f5fce29 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:11.701051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51ab02a7-40ff-41bb-8ae4-f4012f5fce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:11.757254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174e9e4-6588-454c-8cac-e16df2b703c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:11.758307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174e9e4-6588-454c-8cac-e16df2b703c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:11.758798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174e9e4-6588-454c-8cac-e16df2b703c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:11.759427 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174e9e4-6588-454c-8cac-e16df2b703c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:11.760375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174e9e4-6588-454c-8cac-e16df2b703c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:11.762119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174e9e4-6588-454c-8cac-e16df2b703c8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:11.773085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 900/1803] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:11.777209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174e9e4-6588-454c-8cac-e16df2b703c8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:11.788143 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174e9e4-6588-454c-8cac-e16df2b703c8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:11.796308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 904/1804] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:11.796483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f757a8a-657b-4231-b09e-d90a907fa795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:11.796483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f757a8a-657b-4231-b09e-d90a907fa795 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:11.796483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f757a8a-657b-4231-b09e-d90a907fa795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:11.796483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f757a8a-657b-4231-b09e-d90a907fa795 admin admin] PATH_INFO: `/v3/projects/60c6387c01704120a9aa52f38deffb87/users/7df01e838c834005ac869461375d2c9a/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:11.800574 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f757a8a-657b-4231-b09e-d90a907fa795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c6387c01704120a9aa52f38deffb87, user_id=7df01e838c834005ac869461375d2c9a, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:11.803970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab02a7-40ff-41bb-8ae4-f4012f5fce29 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:11.804146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f757a8a-657b-4231-b09e-d90a907fa795 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:11.805236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51ab02a7-40ff-41bb-8ae4-f4012f5fce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:11.818529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 901/1805] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:11 2022] PUT /identity/v3/projects/60c6387c01704120a9aa52f38deffb87/users/7df01e838c834005ac869461375d2c9a/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:11.834529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1863ad-e446-4546-b283-58f879379288 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:11.834529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1863ad-e446-4546-b283-58f879379288 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:11.834529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1863ad-e446-4546-b283-58f879379288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:11.869805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-da1863ad-e446-4546-b283-58f879379288 None None] MFA Rules not processed for user `7df01e838c834005ac869461375d2c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:11.871499 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-da1863ad-e446-4546-b283-58f879379288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:11.878173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:11.883821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab02a7-40ff-41bb-8ae4-f4012f5fce29 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:11.956703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ca197-4d74-48d8-9da9-c93dda68e341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:11.957320 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ca197-4d74-48d8-9da9-c93dda68e341 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:11.957571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ca197-4d74-48d8-9da9-c93dda68e341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:11.958093 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ca197-4d74-48d8-9da9-c93dda68e341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:11.971366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 902/1806] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:11.983128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1863ad-e446-4546-b283-58f879379288 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:11.985226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-da1863ad-e446-4546-b283-58f879379288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:12.051094 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1863ad-e446-4546-b283-58f879379288 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:12.055661 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94f716-1f60-4cbb-89e7-91fa97c8d7da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:12.068327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94f716-1f60-4cbb-89e7-91fa97c8d7da service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:12.068327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94f716-1f60-4cbb-89e7-91fa97c8d7da service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:12.068327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94f716-1f60-4cbb-89e7-91fa97c8d7da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:12.068617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ca197-4d74-48d8-9da9-c93dda68e341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:12.068617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ca197-4d74-48d8-9da9-c93dda68e341 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:12.068770 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94f716-1f60-4cbb-89e7-91fa97c8d7da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:12.069744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94f716-1f60-4cbb-89e7-91fa97c8d7da service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:12.072355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 905/1807] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:12.083909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 903/1808] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:42:12.151860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27b3b-87bc-478d-94b7-a5b023b628a4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:12.152367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27b3b-87bc-478d-94b7-a5b023b628a4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:12.152657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27b3b-87bc-478d-94b7-a5b023b628a4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:12.154464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 906/1809] 10.222.0.64 () {54 vars in 912 bytes} [Thu Aug 4 07:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:42:12.167185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3623e5-5478-4e08-9de2-fbca69e8aadb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:12.167389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3623e5-5478-4e08-9de2-fbca69e8aadb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:12.174395 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3623e5-5478-4e08-9de2-fbca69e8aadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:12.200513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9b3623e5-5478-4e08-9de2-fbca69e8aadb None None] MFA Rules not processed for user `b80e701a81cb4fa893e55288559010f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:12.201870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9b3623e5-5478-4e08-9de2-fbca69e8aadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:12.209020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 904/1810] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:42:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:12.227167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e27b3b-87bc-478d-94b7-a5b023b628a4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:12.228890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a7e27b3b-87bc-478d-94b7-a5b023b628a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:12.244194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e27b3b-87bc-478d-94b7-a5b023b628a4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:12.247768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df489386-2ff2-4d76-8b53-bc3ed3b99605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:12.248422 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df489386-2ff2-4d76-8b53-bc3ed3b99605 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:12.254469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df489386-2ff2-4d76-8b53-bc3ed3b99605 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:12.254469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df489386-2ff2-4d76-8b53-bc3ed3b99605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:12.256321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df489386-2ff2-4d76-8b53-bc3ed3b99605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:12.257749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df489386-2ff2-4d76-8b53-bc3ed3b99605 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:12.266273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 907/1811] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:12.299185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3623e5-5478-4e08-9de2-fbca69e8aadb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:12.311430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3623e5-5478-4e08-9de2-fbca69e8aadb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:12.315970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bd569-58a4-4b78-97da-a22a7e234335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:12.316753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bd569-58a4-4b78-97da-a22a7e234335 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:12.317140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bd569-58a4-4b78-97da-a22a7e234335 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:12.317544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bd569-58a4-4b78-97da-a22a7e234335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:12.320804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9f2bd569-58a4-4b78-97da-a22a7e234335 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:12.408805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bd569-58a4-4b78-97da-a22a7e234335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:12.410530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bd569-58a4-4b78-97da-a22a7e234335 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:12.424339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 905/1812] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:12.595811 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:12.663456 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df489386-2ff2-4d76-8b53-bc3ed3b99605 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:12.680357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-df489386-2ff2-4d76-8b53-bc3ed3b99605 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:12.704917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df489386-2ff2-4d76-8b53-bc3ed3b99605 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:12.716343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:12.718158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:12.718933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bd569-58a4-4b78-97da-a22a7e234335 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:12.719052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:12.719923 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] PATH_INFO: `/v3/projects/96c4257a34cf4ffd89124cfe7511e56f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:12.720650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9f2bd569-58a4-4b78-97da-a22a7e234335 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:12.736525 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c4257a34cf4ffd89124cfe7511e56f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:12.739267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:12.880022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bd569-58a4-4b78-97da-a22a7e234335 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:12.886784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9c474-14c6-45d8-b10d-0066a8d657ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:12.887601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9c474-14c6-45d8-b10d-0066a8d657ac service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:12.887969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9c474-14c6-45d8-b10d-0066a8d657ac service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:12.894197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9c474-14c6-45d8-b10d-0066a8d657ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:12.894197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c7d9c474-14c6-45d8-b10d-0066a8d657ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:12.912504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9c474-14c6-45d8-b10d-0066a8d657ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:12.914261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9c474-14c6-45d8-b10d-0066a8d657ac service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:12.936985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 906/1813] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:42:12.944493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c4257a34cf4ffd89124cfe7511e56f', 'request_id': 'req-6d671de9-d804-43f5-8096-1687e1923927'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:12.953993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c4257a34cf4ffd89124cfe7511e56f', 'request_id': 'req-6d671de9-d804-43f5-8096-1687e1923927'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:13.042719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c4257a34cf4ffd89124cfe7511e56f', 'request_id': 'req-6d671de9-d804-43f5-8096-1687e1923927'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:13.062896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] The token cache is being invalidate because project 96c4257a34cf4ffd89124cfe7511e56f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:13.063420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:13.064466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:13.092336 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 908/1814] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:12 2022] DELETE /identity/v3/projects/96c4257a34cf4ffd89124cfe7511e56f => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:13.100585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9c474-14c6-45d8-b10d-0066a8d657ac service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:13.135439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9c474-14c6-45d8-b10d-0066a8d657ac service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:13.226193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df7292-f26f-4e9a-8438-4f6e52121cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:13.226193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df7292-f26f-4e9a-8438-4f6e52121cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:13.226193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df7292-f26f-4e9a-8438-4f6e52121cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:13.226193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df7292-f26f-4e9a-8438-4f6e52121cd0 admin admin] PATH_INFO: `/v3/users/8786ad461bca4b03a1bdff82e0605db4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:13.226569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df7292-f26f-4e9a-8438-4f6e52121cd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8786ad461bca4b03a1bdff82e0605db4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:13.228067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df7292-f26f-4e9a-8438-4f6e52121cd0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:13.426184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-70df7292-f26f-4e9a-8438-4f6e52121cd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8786ad461bca4b03a1bdff82e0605db4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:13.426935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-70df7292-f26f-4e9a-8438-4f6e52121cd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8786ad461bca4b03a1bdff82e0605db4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:13.433748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-70df7292-f26f-4e9a-8438-4f6e52121cd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8786ad461bca4b03a1bdff82e0605db4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:13.446582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-70df7292-f26f-4e9a-8438-4f6e52121cd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8786ad461bca4b03a1bdff82e0605db4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:13.552581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-70df7292-f26f-4e9a-8438-4f6e52121cd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8786ad461bca4b03a1bdff82e0605db4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:13.583977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 907/1815] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:13 2022] DELETE /identity/v3/users/8786ad461bca4b03a1bdff82e0605db4 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:14.217857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:14.219704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:14.254112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d671de9-d804-43f5-8096-1687e1923927 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:14.300323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:14.377931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:14.398354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:14.399153 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:14.399779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:14.400102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] PATH_INFO: `/v3/projects/d4dcdbb3710f4c31816fdf842d008e8e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:14.402265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4dcdbb3710f4c31816fdf842d008e8e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:14.404852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:14.529966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4dcdbb3710f4c31816fdf842d008e8e', 'request_id': 'req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:14.539017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4dcdbb3710f4c31816fdf842d008e8e', 'request_id': 'req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:14.640285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4dcdbb3710f4c31816fdf842d008e8e', 'request_id': 'req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:14.657420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] The token cache is being invalidate because project d4dcdbb3710f4c31816fdf842d008e8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:14.657913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:14.658853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4bae534-fd32-4ab0-b9f1-18b8cf9ca6c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:14.664964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 909/1816] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:14 2022] DELETE /identity/v3/projects/d4dcdbb3710f4c31816fdf842d008e8e => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:14.666238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43230027-43d7-41ce-b529-1b5f330a5d52 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:14.666679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43230027-43d7-41ce-b529-1b5f330a5d52 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:14.667543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43230027-43d7-41ce-b529-1b5f330a5d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:14.698393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-43230027-43d7-41ce-b529-1b5f330a5d52 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:14.700006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-43230027-43d7-41ce-b529-1b5f330a5d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:14.789593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 908/1817] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:14.796927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4860e-7e82-43b7-a7e8-40bc6525bd2d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:14.799671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4860e-7e82-43b7-a7e8-40bc6525bd2d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:14.799671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4860e-7e82-43b7-a7e8-40bc6525bd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:14.824092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e5b4860e-7e82-43b7-a7e8-40bc6525bd2d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:14.826308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e5b4860e-7e82-43b7-a7e8-40bc6525bd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:14.835659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 910/1818] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:14.845530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43230027-43d7-41ce-b529-1b5f330a5d52 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:14.855278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43230027-43d7-41ce-b529-1b5f330a5d52 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:14.860301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dcc0c-3689-448e-bda7-5afecbfa4ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:14.864983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dcc0c-3689-448e-bda7-5afecbfa4ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:14.864983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dcc0c-3689-448e-bda7-5afecbfa4ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:14.864983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dcc0c-3689-448e-bda7-5afecbfa4ca5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:14.864983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528dcc0c-3689-448e-bda7-5afecbfa4ca5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:14.864983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528dcc0c-3689-448e-bda7-5afecbfa4ca5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:14.864983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528dcc0c-3689-448e-bda7-5afecbfa4ca5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:14.880183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 909/1819] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:42:14.882060 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4860e-7e82-43b7-a7e8-40bc6525bd2d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:14.892558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4860e-7e82-43b7-a7e8-40bc6525bd2d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:14.897340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b92432-2e67-45cc-9289-021ebe7d63f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:14.897973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b92432-2e67-45cc-9289-021ebe7d63f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:14.900391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b92432-2e67-45cc-9289-021ebe7d63f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:14.900755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b92432-2e67-45cc-9289-021ebe7d63f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:14.902191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b92432-2e67-45cc-9289-021ebe7d63f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:14.904734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b92432-2e67-45cc-9289-021ebe7d63f5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:15.028744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 911/1820] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:14 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:15.037909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dcc0c-3689-448e-bda7-5afecbfa4ca5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:15.041027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dcc0c-3689-448e-bda7-5afecbfa4ca5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:15.119998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd27838-6d7f-4e3f-b755-57d884539ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:15.124103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd27838-6d7f-4e3f-b755-57d884539ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:15.124103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd27838-6d7f-4e3f-b755-57d884539ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:15.124103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd27838-6d7f-4e3f-b755-57d884539ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:15.124103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd27838-6d7f-4e3f-b755-57d884539ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:15.125365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd27838-6d7f-4e3f-b755-57d884539ee9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:15.270115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 910/1821] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:15 2022] POST /identity/v3/users => generated 288 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:15.273903 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b92432-2e67-45cc-9289-021ebe7d63f5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:15.287685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b92432-2e67-45cc-9289-021ebe7d63f5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:15.300334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea802c64-131c-4ed9-b2ac-d461a7cf7309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:15.301309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea802c64-131c-4ed9-b2ac-d461a7cf7309 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:15.303472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea802c64-131c-4ed9-b2ac-d461a7cf7309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:15.303975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea802c64-131c-4ed9-b2ac-d461a7cf7309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:15.306848 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea802c64-131c-4ed9-b2ac-d461a7cf7309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:15.309429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea802c64-131c-4ed9-b2ac-d461a7cf7309 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:15.341589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 912/1822] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:15.346416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd27838-6d7f-4e3f-b755-57d884539ee9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:15.370096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd27838-6d7f-4e3f-b755-57d884539ee9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:15.377699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc45ac-f848-48bb-8f2f-e35ccd8bb652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:15.378330 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc45ac-f848-48bb-8f2f-e35ccd8bb652 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:15.378886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc45ac-f848-48bb-8f2f-e35ccd8bb652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:15.378886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc45ac-f848-48bb-8f2f-e35ccd8bb652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:15.379553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc45ac-f848-48bb-8f2f-e35ccd8bb652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:15.381213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc45ac-f848-48bb-8f2f-e35ccd8bb652 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:15.392356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 911/1823] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:15.396801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea802c64-131c-4ed9-b2ac-d461a7cf7309 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:15.407786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea802c64-131c-4ed9-b2ac-d461a7cf7309 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:15.412635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd342ff-4085-4475-a876-d2cf5c939cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:15.413329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd342ff-4085-4475-a876-d2cf5c939cff admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:15.413615 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd342ff-4085-4475-a876-d2cf5c939cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:15.414011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd342ff-4085-4475-a876-d2cf5c939cff admin admin] PATH_INFO: `/v3/projects/40675c49c8ae4a1d844e2702630a8d25/users/0b9e8486eec34deba9bc024067fc2524/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:15.441135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:15.445898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd342ff-4085-4475-a876-d2cf5c939cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=40675c49c8ae4a1d844e2702630a8d25, user_id=0b9e8486eec34deba9bc024067fc2524, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:15.448750 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd342ff-4085-4475-a876-d2cf5c939cff admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:15.558839 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 913/1824] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:15 2022] PUT /identity/v3/projects/40675c49c8ae4a1d844e2702630a8d25/users/0b9e8486eec34deba9bc024067fc2524/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:15.560892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eae11-ac30-4868-8cf9-2ecfcbb69ea5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:15.562251 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eae11-ac30-4868-8cf9-2ecfcbb69ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:15.563420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eae11-ac30-4868-8cf9-2ecfcbb69ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:15.628556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-824eae11-ac30-4868-8cf9-2ecfcbb69ea5 None None] MFA Rules not processed for user `0b9e8486eec34deba9bc024067fc2524`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:15.630399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-824eae11-ac30-4868-8cf9-2ecfcbb69ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:15.747900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 912/1825] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:15.753759 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd342ff-4085-4475-a876-d2cf5c939cff admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:15.983377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd342ff-4085-4475-a876-d2cf5c939cff admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:15.991233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed11a7-21c8-48e1-ade4-96fac87fb5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:15.992355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed11a7-21c8-48e1-ade4-96fac87fb5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:15.992841 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed11a7-21c8-48e1-ade4-96fac87fb5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:15.994916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed11a7-21c8-48e1-ade4-96fac87fb5d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:15.995933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed11a7-21c8-48e1-ade4-96fac87fb5d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:15.999236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed11a7-21c8-48e1-ade4-96fac87fb5d1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:16.057929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193e1eb-ae28-4035-955e-0cdd3c57b5e1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:16.058560 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193e1eb-ae28-4035-955e-0cdd3c57b5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:16.059124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193e1eb-ae28-4035-955e-0cdd3c57b5e1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:16.061491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 913/1826] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:42:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:42:16.079803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a6484-8207-448d-b813-cead8bcf7f1c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:16.080565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a6484-8207-448d-b813-cead8bcf7f1c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:16.080565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a6484-8207-448d-b813-cead8bcf7f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:16.097715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 914/1827] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:15 2022] POST /identity/v3/projects => generated 377 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:16.099275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed11a7-21c8-48e1-ade4-96fac87fb5d1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:16.148096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-938a6484-8207-448d-b813-cead8bcf7f1c None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:16.149343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-938a6484-8207-448d-b813-cead8bcf7f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:16.232623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed11a7-21c8-48e1-ade4-96fac87fb5d1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:16.244260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b7ebc-332f-48fe-96c0-bd809a355bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:16.245849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b7ebc-332f-48fe-96c0-bd809a355bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:16.246747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b7ebc-332f-48fe-96c0-bd809a355bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:16.247033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b7ebc-332f-48fe-96c0-bd809a355bdd admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:16.247743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b7ebc-332f-48fe-96c0-bd809a355bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:16.254901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b7ebc-332f-48fe-96c0-bd809a355bdd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:16.356014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 914/1828] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:42:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:16.403664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 915/1829] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:16 2022] POST /identity/v3/users => generated 295 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:16.408795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a6484-8207-448d-b813-cead8bcf7f1c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:16.416555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b7ebc-332f-48fe-96c0-bd809a355bdd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:16.419302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-113b7ebc-332f-48fe-96c0-bd809a355bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:16.429973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a6484-8207-448d-b813-cead8bcf7f1c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:16.463000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54bd5e-9212-4f7e-820f-929f72dda382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:16.463775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54bd5e-9212-4f7e-820f-929f72dda382 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:16.464086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54bd5e-9212-4f7e-820f-929f72dda382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:16.464364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54bd5e-9212-4f7e-820f-929f72dda382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:16.470629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54bd5e-9212-4f7e-820f-929f72dda382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:16.477643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54bd5e-9212-4f7e-820f-929f72dda382 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:16.505265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b7ebc-332f-48fe-96c0-bd809a355bdd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:16.518583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 915/1830] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:16.526559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2c17b-6c1e-42fb-b80f-17cf6a38d4cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:16.528751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2c17b-6c1e-42fb-b80f-17cf6a38d4cd service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:16.529257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2c17b-6c1e-42fb-b80f-17cf6a38d4cd service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:16.529720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2c17b-6c1e-42fb-b80f-17cf6a38d4cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:16.531299 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54bd5e-9212-4f7e-820f-929f72dda382 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:16.567913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2c17b-6c1e-42fb-b80f-17cf6a38d4cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:16.569658 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2c17b-6c1e-42fb-b80f-17cf6a38d4cd service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:16.581769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54bd5e-9212-4f7e-820f-929f72dda382 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:16.619976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d81e14-04c7-4c29-a3e4-64562d9491ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:16.620470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d81e14-04c7-4c29-a3e4-64562d9491ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:16.620751 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d81e14-04c7-4c29-a3e4-64562d9491ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:16.621026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d81e14-04c7-4c29-a3e4-64562d9491ef admin admin] PATH_INFO: `/v3/projects/b34f5f8ac78447fdbb185be350341193/users/d38ad46f144f462eb45b915e2087ca9d/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:16.630597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 916/1831] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:16.655542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d81e14-04c7-4c29-a3e4-64562d9491ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34f5f8ac78447fdbb185be350341193, user_id=d38ad46f144f462eb45b915e2087ca9d, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:16.667721 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d81e14-04c7-4c29-a3e4-64562d9491ef admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:16.790333 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 916/1832] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:16 2022] PUT /identity/v3/projects/b34f5f8ac78447fdbb185be350341193/users/d38ad46f144f462eb45b915e2087ca9d/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:16.794762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2c17b-6c1e-42fb-b80f-17cf6a38d4cd service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:16.809480 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2c17b-6c1e-42fb-b80f-17cf6a38d4cd service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:16.905626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35cb88-7164-4a24-8a26-b69240e0e9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:16.906382 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35cb88-7164-4a24-8a26-b69240e0e9bc admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:16.906504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35cb88-7164-4a24-8a26-b69240e0e9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:16.906766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35cb88-7164-4a24-8a26-b69240e0e9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:16.918122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35cb88-7164-4a24-8a26-b69240e0e9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:16.920021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35cb88-7164-4a24-8a26-b69240e0e9bc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:16.933534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 917/1833] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:16.938787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d81e14-04c7-4c29-a3e4-64562d9491ef admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:16.951407 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d81e14-04c7-4c29-a3e4-64562d9491ef admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:16.967543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c15639-0c95-4284-96bf-d366b09851eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:16.969777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c15639-0c95-4284-96bf-d366b09851eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:16.970143 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c15639-0c95-4284-96bf-d366b09851eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:16.971126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c15639-0c95-4284-96bf-d366b09851eb admin admin] PATH_INFO: `/v3/projects/b34f5f8ac78447fdbb185be350341193/users/d38ad46f144f462eb45b915e2087ca9d/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:16.975921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c15639-0c95-4284-96bf-d366b09851eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34f5f8ac78447fdbb185be350341193, user_id=d38ad46f144f462eb45b915e2087ca9d, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:16.979348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c15639-0c95-4284-96bf-d366b09851eb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:16.996654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 917/1834] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:16 2022] PUT /identity/v3/projects/b34f5f8ac78447fdbb185be350341193/users/d38ad46f144f462eb45b915e2087ca9d/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:17.010847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b50c47-da74-470e-9b04-b0481df07190 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:17.010847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b50c47-da74-470e-9b04-b0481df07190 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:17.010847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b50c47-da74-470e-9b04-b0481df07190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:17.061841 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c3b50c47-da74-470e-9b04-b0481df07190 None None] MFA Rules not processed for user `d38ad46f144f462eb45b915e2087ca9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:17.064648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c3b50c47-da74-470e-9b04-b0481df07190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:17.194766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 918/1835] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:17.210419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:17.525424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c15639-0c95-4284-96bf-d366b09851eb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:17.526221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-48c15639-0c95-4284-96bf-d366b09851eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:17.650350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c15639-0c95-4284-96bf-d366b09851eb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:17.654289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a7fa3-0d0c-4572-98f6-91c3bf5237ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:17.655244 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a7fa3-0d0c-4572-98f6-91c3bf5237ab service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:17.655644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a7fa3-0d0c-4572-98f6-91c3bf5237ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:17.656781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a7fa3-0d0c-4572-98f6-91c3bf5237ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:17.741600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a7fa3-0d0c-4572-98f6-91c3bf5237ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:17.743661 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a7fa3-0d0c-4572-98f6-91c3bf5237ab service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:17.755727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 918/1836] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:18.094909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:19.484816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:22.056978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f409521-6bb7-437e-be76-6dd5a10f67b4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:22.057597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f409521-6bb7-437e-be76-6dd5a10f67b4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:22.057597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f409521-6bb7-437e-be76-6dd5a10f67b4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:22.059940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 919/1837] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:42:22.071923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d10c3-54b0-4cd2-9f38-f21552f7242d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:22.072494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d10c3-54b0-4cd2-9f38-f21552f7242d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:22.072701 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d10c3-54b0-4cd2-9f38-f21552f7242d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:22.097633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-458d10c3-54b0-4cd2-9f38-f21552f7242d None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:22.098927 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-458d10c3-54b0-4cd2-9f38-f21552f7242d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:22.189889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 919/1838] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:42:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:22.201531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f409521-6bb7-437e-be76-6dd5a10f67b4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:22.219178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f409521-6bb7-437e-be76-6dd5a10f67b4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:22.225020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6ee77-e712-4089-83d7-1bee439b5708 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:22.225643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6ee77-e712-4089-83d7-1bee439b5708 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:22.225928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6ee77-e712-4089-83d7-1bee439b5708 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:22.226206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6ee77-e712-4089-83d7-1bee439b5708 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:22.237703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6ee77-e712-4089-83d7-1bee439b5708 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:22.238340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6ee77-e712-4089-83d7-1bee439b5708 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:22.249729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 920/1839] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:26.782117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d10c3-54b0-4cd2-9f38-f21552f7242d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:26.791867 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d10c3-54b0-4cd2-9f38-f21552f7242d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:26.797970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3457569-2743-4bf0-b813-8959abdc756f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:26.798896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3457569-2743-4bf0-b813-8959abdc756f service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:26.799251 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3457569-2743-4bf0-b813-8959abdc756f service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:26.799642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3457569-2743-4bf0-b813-8959abdc756f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:26.810136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3457569-2743-4bf0-b813-8959abdc756f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:26.812016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3457569-2743-4bf0-b813-8959abdc756f service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:26.822376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 920/1840] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:42:28.576426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6ee77-e712-4089-83d7-1bee439b5708 service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:28.578524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a3b6ee77-e712-4089-83d7-1bee439b5708 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:28.644005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6ee77-e712-4089-83d7-1bee439b5708 service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:28.648586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88cb75-3915-43c1-a6ac-134ed22d7771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:28.649226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88cb75-3915-43c1-a6ac-134ed22d7771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:28.649513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88cb75-3915-43c1-a6ac-134ed22d7771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:28.649794 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88cb75-3915-43c1-a6ac-134ed22d7771 admin admin] PATH_INFO: `/v3/users/68119d036cbd415eba57f1774a248a9f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:28.650948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88cb75-3915-43c1-a6ac-134ed22d7771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68119d036cbd415eba57f1774a248a9f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:28.652969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88cb75-3915-43c1-a6ac-134ed22d7771 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:28.911616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-da88cb75-3915-43c1-a6ac-134ed22d7771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68119d036cbd415eba57f1774a248a9f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:28.912364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-da88cb75-3915-43c1-a6ac-134ed22d7771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68119d036cbd415eba57f1774a248a9f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:28.913103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-da88cb75-3915-43c1-a6ac-134ed22d7771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68119d036cbd415eba57f1774a248a9f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:28.926471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-da88cb75-3915-43c1-a6ac-134ed22d7771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68119d036cbd415eba57f1774a248a9f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:28.971425 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3457569-2743-4bf0-b813-8959abdc756f service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:28.972898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c3457569-2743-4bf0-b813-8959abdc756f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:28.986861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3457569-2743-4bf0-b813-8959abdc756f service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:28.991302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:28.991956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:28.992272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:28.992552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] PATH_INFO: `/v3/users/718253b715c245c5907acb407d5112e4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:28.993695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=718253b715c245c5907acb407d5112e4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:28.995605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:29.000766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-da88cb75-3915-43c1-a6ac-134ed22d7771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68119d036cbd415eba57f1774a248a9f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:29.008314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:29.020043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 921/1841] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:28 2022] DELETE /identity/v3/users/68119d036cbd415eba57f1774a248a9f => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:29.025407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684ff79-17f4-4780-b9e9-cea4d43ba7c3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:29.025707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684ff79-17f4-4780-b9e9-cea4d43ba7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:29.025988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684ff79-17f4-4780-b9e9-cea4d43ba7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:29.045251 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3684ff79-17f4-4780-b9e9-cea4d43ba7c3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:29.046939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3684ff79-17f4-4780-b9e9-cea4d43ba7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:29.079721 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:29.105416 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 922/1842] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:29.115601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684ff79-17f4-4780-b9e9-cea4d43ba7c3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:29.116976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3684ff79-17f4-4780-b9e9-cea4d43ba7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:29.174069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684ff79-17f4-4780-b9e9-cea4d43ba7c3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:29.178172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb2795-7e0a-43d6-909c-6e7eed9d4b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:29.178978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb2795-7e0a-43d6-909c-6e7eed9d4b3e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:29.179323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb2795-7e0a-43d6-909c-6e7eed9d4b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:29.179641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb2795-7e0a-43d6-909c-6e7eed9d4b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:29.189269 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb2795-7e0a-43d6-909c-6e7eed9d4b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:29.190718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb2795-7e0a-43d6-909c-6e7eed9d4b3e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:29.200318 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 923/1843] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:42:29.241120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '718253b715c245c5907acb407d5112e4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:29.242767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '718253b715c245c5907acb407d5112e4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:29.261950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '718253b715c245c5907acb407d5112e4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:29.287586 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '718253b715c245c5907acb407d5112e4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:29.360568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '718253b715c245c5907acb407d5112e4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:29.373465 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 921/1844] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:28 2022] DELETE /identity/v3/users/718253b715c245c5907acb407d5112e4 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:29.373603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94215e23-896b-4f9b-9606-1cb3012745c8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:29.373896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94215e23-896b-4f9b-9606-1cb3012745c8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:29.374696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94215e23-896b-4f9b-9606-1cb3012745c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:29.422868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-94215e23-896b-4f9b-9606-1cb3012745c8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:29.424071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-94215e23-896b-4f9b-9606-1cb3012745c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:29.502566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 924/1845] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:29.512500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:29.513925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:29.587504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30d381-d656-46f4-8027-ceeaa343e329 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:29.591234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9522c1-60b2-4d4c-8768-c8095216f880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:29.592359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9522c1-60b2-4d4c-8768-c8095216f880 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:29.592645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9522c1-60b2-4d4c-8768-c8095216f880 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:29.593584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9522c1-60b2-4d4c-8768-c8095216f880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:29.603040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9522c1-60b2-4d4c-8768-c8095216f880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:29.604517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9522c1-60b2-4d4c-8768-c8095216f880 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:29.615335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 922/1846] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:29.794628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94215e23-896b-4f9b-9606-1cb3012745c8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:29.796667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-94215e23-896b-4f9b-9606-1cb3012745c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:29.809064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94215e23-896b-4f9b-9606-1cb3012745c8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:29.819524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:29.820264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:29.820603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:29.820865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] PATH_INFO: `/v3/projects/85979891703b4e8cb302468f679f0316` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:29.822469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85979891703b4e8cb302468f679f0316)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:29.824464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:29.963243 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85979891703b4e8cb302468f679f0316', 'request_id': 'req-47942ed3-8eb9-486f-911b-885158988996'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:29.971474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85979891703b4e8cb302468f679f0316', 'request_id': 'req-47942ed3-8eb9-486f-911b-885158988996'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.049456 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85979891703b4e8cb302468f679f0316', 'request_id': 'req-47942ed3-8eb9-486f-911b-885158988996'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.061071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] The token cache is being invalidate because project 85979891703b4e8cb302468f679f0316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:30.061360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.062195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.063739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 925/1847] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:29 2022] DELETE /identity/v3/projects/85979891703b4e8cb302468f679f0316 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:30.068061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9522c1-60b2-4d4c-8768-c8095216f880 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:30.077988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9522c1-60b2-4d4c-8768-c8095216f880 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:30.116567 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:30.118077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:30.129658 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:30.143428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:30.144405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:30.145643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:30.145643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] PATH_INFO: `/v3/users/8c6e0d58d5e54fc18b0dc7f88bd4b5c9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:30.146488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6e0d58d5e54fc18b0dc7f88bd4b5c9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:30.148581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:30.196503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47942ed3-8eb9-486f-911b-885158988996 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:30.201190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:30.201835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:30.202125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:30.202433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] PATH_INFO: `/v3/projects/be2daa8d0ac74d9a9a67f77d367da9db` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:30.203617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2daa8d0ac74d9a9a67f77d367da9db)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:30.205565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:30.372110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:30.386827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6e0d58d5e54fc18b0dc7f88bd4b5c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.387488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2daa8d0ac74d9a9a67f77d367da9db', 'request_id': 'req-fddb3796-be8a-4953-9a25-1b44545f7b5c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.387609 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6e0d58d5e54fc18b0dc7f88bd4b5c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.394422 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2daa8d0ac74d9a9a67f77d367da9db', 'request_id': 'req-fddb3796-be8a-4953-9a25-1b44545f7b5c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.395006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6e0d58d5e54fc18b0dc7f88bd4b5c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.407312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6e0d58d5e54fc18b0dc7f88bd4b5c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.451370 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2daa8d0ac74d9a9a67f77d367da9db', 'request_id': 'req-fddb3796-be8a-4953-9a25-1b44545f7b5c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.464050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] The token cache is being invalidate because project be2daa8d0ac74d9a9a67f77d367da9db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:30.464348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.465040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.466582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 926/1848] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:30 2022] DELETE /identity/v3/projects/be2daa8d0ac74d9a9a67f77d367da9db => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:30.471312 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:30.472922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:30.513866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6e0d58d5e54fc18b0dc7f88bd4b5c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:30.523806 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 923/1849] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:30 2022] DELETE /identity/v3/users/8c6e0d58d5e54fc18b0dc7f88bd4b5c9 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:30.546497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb3796-be8a-4953-9a25-1b44545f7b5c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:30.602052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:30.622369 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641882de-a8a4-44bb-94a9-ba2816b671c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:30.623020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641882de-a8a4-44bb-94a9-ba2816b671c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:30.623304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641882de-a8a4-44bb-94a9-ba2816b671c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:30.623585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641882de-a8a4-44bb-94a9-ba2816b671c9 admin admin] PATH_INFO: `/v3/users/7df01e838c834005ac869461375d2c9a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:30.625262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641882de-a8a4-44bb-94a9-ba2816b671c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df01e838c834005ac869461375d2c9a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:30.627165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641882de-a8a4-44bb-94a9-ba2816b671c9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:31.045730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-641882de-a8a4-44bb-94a9-ba2816b671c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df01e838c834005ac869461375d2c9a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.047165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-641882de-a8a4-44bb-94a9-ba2816b671c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df01e838c834005ac869461375d2c9a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.047812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-641882de-a8a4-44bb-94a9-ba2816b671c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df01e838c834005ac869461375d2c9a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.081255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-641882de-a8a4-44bb-94a9-ba2816b671c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df01e838c834005ac869461375d2c9a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.131253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-641882de-a8a4-44bb-94a9-ba2816b671c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df01e838c834005ac869461375d2c9a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.144889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 927/1850] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:30 2022] DELETE /identity/v3/users/7df01e838c834005ac869461375d2c9a => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:31.160728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:31.162579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:31.178078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cb118-b475-4253-9ca2-a26336088475 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:31.242967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:31.243718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:31.244208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:31.244491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] PATH_INFO: `/v3/projects/5918d40605d84f1fab05e39ae4a59d9c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:31.245674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5918d40605d84f1fab05e39ae4a59d9c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:31.247593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:31.398339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5918d40605d84f1fab05e39ae4a59d9c', 'request_id': 'req-814ddaa5-b767-4afe-ba4c-90a23a9821d4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.479554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5918d40605d84f1fab05e39ae4a59d9c', 'request_id': 'req-814ddaa5-b767-4afe-ba4c-90a23a9821d4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.488131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5918d40605d84f1fab05e39ae4a59d9c', 'request_id': 'req-814ddaa5-b767-4afe-ba4c-90a23a9821d4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.500673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] The token cache is being invalidate because project 5918d40605d84f1fab05e39ae4a59d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:31.501039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.501757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-814ddaa5-b767-4afe-ba4c-90a23a9821d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.503340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 924/1851] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:31 2022] DELETE /identity/v3/projects/5918d40605d84f1fab05e39ae4a59d9c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:31.510202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52488a6f-2827-4331-a583-27d588c245fc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:31.517167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52488a6f-2827-4331-a583-27d588c245fc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:31.517167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52488a6f-2827-4331-a583-27d588c245fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:31.531546 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-52488a6f-2827-4331-a583-27d588c245fc None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:31.532743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-52488a6f-2827-4331-a583-27d588c245fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:31.597351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 928/1852] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:31.604757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5832-a635-4ab8-9a36-fc6ee74a639e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:31.605207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5832-a635-4ab8-9a36-fc6ee74a639e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:31.605387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5832-a635-4ab8-9a36-fc6ee74a639e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:31.610921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52488a6f-2827-4331-a583-27d588c245fc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:31.612541 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-52488a6f-2827-4331-a583-27d588c245fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:31.632577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52488a6f-2827-4331-a583-27d588c245fc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:31.637420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:31.639018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:31.639775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:31.640363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] PATH_INFO: `/v3/projects/60c6387c01704120a9aa52f38deffb87` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:31.641951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=60c6387c01704120a9aa52f38deffb87)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:31.645612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:31.649086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-06af5832-a635-4ab8-9a36-fc6ee74a639e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:31.649455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-06af5832-a635-4ab8-9a36-fc6ee74a639e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:31.659751 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 925/1853] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:31.663824 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5832-a635-4ab8-9a36-fc6ee74a639e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:31.682528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5832-a635-4ab8-9a36-fc6ee74a639e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:31.688280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11794d5-7587-4a5e-9df4-1a162fdd7f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:31.688907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11794d5-7587-4a5e-9df4-1a162fdd7f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:31.689189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11794d5-7587-4a5e-9df4-1a162fdd7f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:31.690377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11794d5-7587-4a5e-9df4-1a162fdd7f00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:31.690377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11794d5-7587-4a5e-9df4-1a162fdd7f00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:31.690532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11794d5-7587-4a5e-9df4-1a162fdd7f00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:31.692646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11794d5-7587-4a5e-9df4-1a162fdd7f00 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:31.706174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 926/1854] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:42:31.711034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11794d5-7587-4a5e-9df4-1a162fdd7f00 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:31.728410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11794d5-7587-4a5e-9df4-1a162fdd7f00 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:31.728410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8af9a-4a2a-421d-a7f6-20b68c74689b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:31.728410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8af9a-4a2a-421d-a7f6-20b68c74689b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:31.728410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8af9a-4a2a-421d-a7f6-20b68c74689b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:31.729018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8af9a-4a2a-421d-a7f6-20b68c74689b admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:31.729018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8af9a-4a2a-421d-a7f6-20b68c74689b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:31.730769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8af9a-4a2a-421d-a7f6-20b68c74689b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:31.748289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:31.793176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60c6387c01704120a9aa52f38deffb87', 'request_id': 'req-862484ad-902f-4ea8-87dd-f5c57f8548ac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.801312 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60c6387c01704120a9aa52f38deffb87', 'request_id': 'req-862484ad-902f-4ea8-87dd-f5c57f8548ac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.816642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:31.864803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 927/1855] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:31 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:31.865801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8af9a-4a2a-421d-a7f6-20b68c74689b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:31.914873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60c6387c01704120a9aa52f38deffb87', 'request_id': 'req-862484ad-902f-4ea8-87dd-f5c57f8548ac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.932216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] The token cache is being invalidate because project 60c6387c01704120a9aa52f38deffb87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:31.932216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.932216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-862484ad-902f-4ea8-87dd-f5c57f8548ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:31.936464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 929/1856] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:31 2022] DELETE /identity/v3/projects/60c6387c01704120a9aa52f38deffb87 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:31.938143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0b268-50df-4e78-a576-379cee2d71bc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:31.938542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0b268-50df-4e78-a576-379cee2d71bc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:31.938976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0b268-50df-4e78-a576-379cee2d71bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:31.960605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8af9a-4a2a-421d-a7f6-20b68c74689b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:31.967179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6264d9-a2ee-415d-9a7a-1811bebca566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:31.968726 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6264d9-a2ee-415d-9a7a-1811bebca566 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:31.969087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6264d9-a2ee-415d-9a7a-1811bebca566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:31.969505 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6264d9-a2ee-415d-9a7a-1811bebca566 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:31.970545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6264d9-a2ee-415d-9a7a-1811bebca566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:31.971868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c1c0b268-50df-4e78-a576-379cee2d71bc None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:31.973227 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c1c0b268-50df-4e78-a576-379cee2d71bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:31.975803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6264d9-a2ee-415d-9a7a-1811bebca566 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:31.985620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 930/1857] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:32.004824 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058361e3-1c95-43b3-8cf7-c15c13e64a59 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.005205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058361e3-1c95-43b3-8cf7-c15c13e64a59 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.005715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058361e3-1c95-43b3-8cf7-c15c13e64a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.031851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-058361e3-1c95-43b3-8cf7-c15c13e64a59 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:32.033420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-058361e3-1c95-43b3-8cf7-c15c13e64a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:32.042933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 931/1858] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:32.046781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058361e3-1c95-43b3-8cf7-c15c13e64a59 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.100791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 928/1859] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:31 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:32.102807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058361e3-1c95-43b3-8cf7-c15c13e64a59 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.104859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6264d9-a2ee-415d-9a7a-1811bebca566 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.107532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5f6264d9-a2ee-415d-9a7a-1811bebca566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:32.120533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da9744-ea1e-4c6a-a680-78e34cb0bbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.121509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da9744-ea1e-4c6a-a680-78e34cb0bbab admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.121976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da9744-ea1e-4c6a-a680-78e34cb0bbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.122492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da9744-ea1e-4c6a-a680-78e34cb0bbab admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.123429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da9744-ea1e-4c6a-a680-78e34cb0bbab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:32.123956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da9744-ea1e-4c6a-a680-78e34cb0bbab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.125997 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da9744-ea1e-4c6a-a680-78e34cb0bbab admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.131438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6264d9-a2ee-415d-9a7a-1811bebca566 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.145674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56695c19-7346-4cac-ba5b-d89d67b39276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.146374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56695c19-7346-4cac-ba5b-d89d67b39276 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.150684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56695c19-7346-4cac-ba5b-d89d67b39276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.150684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56695c19-7346-4cac-ba5b-d89d67b39276 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.150684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56695c19-7346-4cac-ba5b-d89d67b39276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.150684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56695c19-7346-4cac-ba5b-d89d67b39276 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.158092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 932/1860] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:42:32.170737 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da9744-ea1e-4c6a-a680-78e34cb0bbab admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.179157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 929/1861] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:32.184546 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da9744-ea1e-4c6a-a680-78e34cb0bbab admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.188774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56695c19-7346-4cac-ba5b-d89d67b39276 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.192571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8ba9a-152d-4a1c-938e-ca870153e7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.193386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8ba9a-152d-4a1c-938e-ca870153e7da admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.193855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8ba9a-152d-4a1c-938e-ca870153e7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.194140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8ba9a-152d-4a1c-938e-ca870153e7da admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.194826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8ba9a-152d-4a1c-938e-ca870153e7da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.197425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8ba9a-152d-4a1c-938e-ca870153e7da admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.201126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56695c19-7346-4cac-ba5b-d89d67b39276 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.210903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87cf31-21a0-47eb-a43e-a4db25511ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.212216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87cf31-21a0-47eb-a43e-a4db25511ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.212628 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87cf31-21a0-47eb-a43e-a4db25511ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.212977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87cf31-21a0-47eb-a43e-a4db25511ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.214203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87cf31-21a0-47eb-a43e-a4db25511ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.217053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87cf31-21a0-47eb-a43e-a4db25511ac2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.232879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 930/1862] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:32.240823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87cf31-21a0-47eb-a43e-a4db25511ac2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.256966 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87cf31-21a0-47eb-a43e-a4db25511ac2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.264340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2d317-e529-4c9d-93db-bd5217056b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.265302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2d317-e529-4c9d-93db-bd5217056b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.265912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2d317-e529-4c9d-93db-bd5217056b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.265912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2d317-e529-4c9d-93db-bd5217056b1c admin admin] PATH_INFO: `/v3/projects/56d106a358cd4ac8a4286cc4a0c41c8e/users/fcd145d9bd644f3bb150dfc7d43d1c8a/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.283101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2d317-e529-4c9d-93db-bd5217056b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=56d106a358cd4ac8a4286cc4a0c41c8e, user_id=fcd145d9bd644f3bb150dfc7d43d1c8a, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.285408 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2d317-e529-4c9d-93db-bd5217056b1c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.304813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 933/1863] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:32 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:32.309225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8ba9a-152d-4a1c-938e-ca870153e7da admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.363787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 931/1864] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:32 2022] PUT /identity/v3/projects/56d106a358cd4ac8a4286cc4a0c41c8e/users/fcd145d9bd644f3bb150dfc7d43d1c8a/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:32.374255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d6fda-93f1-42cb-a783-8ee9cdf7473e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.374547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d6fda-93f1-42cb-a783-8ee9cdf7473e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.375156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d6fda-93f1-42cb-a783-8ee9cdf7473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.386743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8ba9a-152d-4a1c-938e-ca870153e7da admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.404383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-876d6fda-93f1-42cb-a783-8ee9cdf7473e None None] MFA Rules not processed for user `fcd145d9bd644f3bb150dfc7d43d1c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:32.407032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-876d6fda-93f1-42cb-a783-8ee9cdf7473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:32.457685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712390cd-86c5-4d16-8455-0758792e9adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.458573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712390cd-86c5-4d16-8455-0758792e9adc admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.458941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712390cd-86c5-4d16-8455-0758792e9adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.459237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712390cd-86c5-4d16-8455-0758792e9adc admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.459909 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712390cd-86c5-4d16-8455-0758792e9adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.464220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712390cd-86c5-4d16-8455-0758792e9adc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.489236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 932/1865] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:32.493238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d6fda-93f1-42cb-a783-8ee9cdf7473e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.504142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d6fda-93f1-42cb-a783-8ee9cdf7473e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.509325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3642d2-4ee9-4542-ac37-643ef773affa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.510066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3642d2-4ee9-4542-ac37-643ef773affa admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.510399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3642d2-4ee9-4542-ac37-643ef773affa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.516054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3642d2-4ee9-4542-ac37-643ef773affa admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.516054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3642d2-4ee9-4542-ac37-643ef773affa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.516054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3642d2-4ee9-4542-ac37-643ef773affa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.559389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 934/1866] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:32 2022] POST /identity/v3/users => generated 285 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:32.562787 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712390cd-86c5-4d16-8455-0758792e9adc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.572598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712390cd-86c5-4d16-8455-0758792e9adc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.577202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7ec3-1414-47de-bcd6-7d627688fa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.577811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7ec3-1414-47de-bcd6-7d627688fa60 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.578099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7ec3-1414-47de-bcd6-7d627688fa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.578380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7ec3-1414-47de-bcd6-7d627688fa60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.579076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada7ec3-1414-47de-bcd6-7d627688fa60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.580702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada7ec3-1414-47de-bcd6-7d627688fa60 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.590631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 935/1867] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:32.595185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7ec3-1414-47de-bcd6-7d627688fa60 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.604596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7ec3-1414-47de-bcd6-7d627688fa60 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.609883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571cd2c-332d-4af0-a9d7-b9e852912e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.610470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571cd2c-332d-4af0-a9d7-b9e852912e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.610753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571cd2c-332d-4af0-a9d7-b9e852912e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.611029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571cd2c-332d-4af0-a9d7-b9e852912e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.611730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571cd2c-332d-4af0-a9d7-b9e852912e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.613315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571cd2c-332d-4af0-a9d7-b9e852912e90 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.623938 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 936/1868] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:32.624526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 933/1869] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:32 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:32.628903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3642d2-4ee9-4542-ac37-643ef773affa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.629069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571cd2c-332d-4af0-a9d7-b9e852912e90 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.639313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571cd2c-332d-4af0-a9d7-b9e852912e90 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.644522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:32.653339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:32.712418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1cbc7-0560-42ad-9b0e-c82c002a600f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.713126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1cbc7-0560-42ad-9b0e-c82c002a600f admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.713230 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3642d2-4ee9-4542-ac37-643ef773affa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.713421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1cbc7-0560-42ad-9b0e-c82c002a600f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.713699 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1cbc7-0560-42ad-9b0e-c82c002a600f admin admin] PATH_INFO: `/v3/projects/851f8f8a6e464673a5cf2b0c047bb1a0/users/55a0d313bb9e4648a53548168614e440/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.726678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167400f-732e-4e08-ac74-0f582705c3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.726678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167400f-732e-4e08-ac74-0f582705c3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.726678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167400f-732e-4e08-ac74-0f582705c3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.727030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167400f-732e-4e08-ac74-0f582705c3d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.727723 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167400f-732e-4e08-ac74-0f582705c3d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.729763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167400f-732e-4e08-ac74-0f582705c3d2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.734045 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1cbc7-0560-42ad-9b0e-c82c002a600f admin admin] RBAC: Authorizing `identity:create_grant(project_id=851f8f8a6e464673a5cf2b0c047bb1a0, user_id=55a0d313bb9e4648a53548168614e440, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.737769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1cbc7-0560-42ad-9b0e-c82c002a600f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.841043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 937/1870] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:32 2022] PUT /identity/v3/projects/851f8f8a6e464673a5cf2b0c047bb1a0/users/55a0d313bb9e4648a53548168614e440/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:32.847196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 934/1871] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:32 2022] POST /identity/v3/users => generated 299 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:32.852411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1cbc7-0560-42ad-9b0e-c82c002a600f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.857945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26d95-a0a5-403d-9c63-fc1ef2ff4b92 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.858129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26d95-a0a5-403d-9c63-fc1ef2ff4b92 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.858628 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26d95-a0a5-403d-9c63-fc1ef2ff4b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.889202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2fb26d95-a0a5-403d-9c63-fc1ef2ff4b92 None None] MFA Rules not processed for user `55a0d313bb9e4648a53548168614e440`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:32.891168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2fb26d95-a0a5-403d-9c63-fc1ef2ff4b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:32.923873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1cbc7-0560-42ad-9b0e-c82c002a600f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.931542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ce2d7-28eb-44a1-9c94-18cb437f3684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.932675 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ce2d7-28eb-44a1-9c94-18cb437f3684 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.933988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ce2d7-28eb-44a1-9c94-18cb437f3684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.934582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ce2d7-28eb-44a1-9c94-18cb437f3684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.935855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ce2d7-28eb-44a1-9c94-18cb437f3684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:32.938384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ce2d7-28eb-44a1-9c94-18cb437f3684 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:32.964052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 938/1872] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:32.965121 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ce2d7-28eb-44a1-9c94-18cb437f3684 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.983256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 935/1873] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:32.983453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ce2d7-28eb-44a1-9c94-18cb437f3684 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:32.986581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26d95-a0a5-403d-9c63-fc1ef2ff4b92 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:32.988872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083e322-2dc8-423b-b08f-4964a41681d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:32.989559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e322-2dc8-423b-b08f-4964a41681d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:32.989932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e322-2dc8-423b-b08f-4964a41681d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:32.990281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e322-2dc8-423b-b08f-4964a41681d2 admin admin] PATH_INFO: `/v3/projects/814055191dd4462ba473eddad263136c/users/af53f488e38e4c8798aa1a38b2b3136b/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:32.998066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26d95-a0a5-403d-9c63-fc1ef2ff4b92 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:33.004863 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a873d-cb96-4b21-b824-1cfb65d7c586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:33.005490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a873d-cb96-4b21-b824-1cfb65d7c586 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:33.005769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a873d-cb96-4b21-b824-1cfb65d7c586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:33.006043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a873d-cb96-4b21-b824-1cfb65d7c586 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:33.006719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a873d-cb96-4b21-b824-1cfb65d7c586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:33.008696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a873d-cb96-4b21-b824-1cfb65d7c586 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:33.011889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083e322-2dc8-423b-b08f-4964a41681d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=814055191dd4462ba473eddad263136c, user_id=af53f488e38e4c8798aa1a38b2b3136b, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:33.011889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083e322-2dc8-423b-b08f-4964a41681d2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:33.079165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 939/1874] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:32 2022] PUT /identity/v3/projects/814055191dd4462ba473eddad263136c/users/af53f488e38e4c8798aa1a38b2b3136b/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:33.083301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083e322-2dc8-423b-b08f-4964a41681d2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:33.105056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 936/1875] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:32 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:33.111292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a873d-cb96-4b21-b824-1cfb65d7c586 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:33.168028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083e322-2dc8-423b-b08f-4964a41681d2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:33.193679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a873d-cb96-4b21-b824-1cfb65d7c586 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:33.198706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d44d67-9f1d-45e4-8e5a-28cd8db09277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:33.199604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d44d67-9f1d-45e4-8e5a-28cd8db09277 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:33.199958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d44d67-9f1d-45e4-8e5a-28cd8db09277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:33.200326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d44d67-9f1d-45e4-8e5a-28cd8db09277 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:33.201251 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d44d67-9f1d-45e4-8e5a-28cd8db09277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:33.204717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d44d67-9f1d-45e4-8e5a-28cd8db09277 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:33.214492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:33.221120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:33.284806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99f374-456f-4627-8e14-480682bee650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:33.285441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99f374-456f-4627-8e14-480682bee650 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:33.285824 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99f374-456f-4627-8e14-480682bee650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:33.286131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99f374-456f-4627-8e14-480682bee650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:33.286894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99f374-456f-4627-8e14-480682bee650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:33.288706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99f374-456f-4627-8e14-480682bee650 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:33.313234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 937/1876] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:33 2022] POST /identity/v3/users => generated 290 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:33.317019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d44d67-9f1d-45e4-8e5a-28cd8db09277 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:33.320516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 940/1877] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:33.327531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99f374-456f-4627-8e14-480682bee650 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:33.348221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d44d67-9f1d-45e4-8e5a-28cd8db09277 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:33.357479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99f374-456f-4627-8e14-480682bee650 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:33.366312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39dd50-6382-4ea5-8022-747584c41071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:33.366965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39dd50-6382-4ea5-8022-747584c41071 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:33.367253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39dd50-6382-4ea5-8022-747584c41071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:33.367534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39dd50-6382-4ea5-8022-747584c41071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:33.368067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14894c0-5960-49c5-a4ce-f8c214b6b010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:33.368328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39dd50-6382-4ea5-8022-747584c41071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:33.368915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14894c0-5960-49c5-a4ce-f8c214b6b010 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:33.369451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14894c0-5960-49c5-a4ce-f8c214b6b010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:33.369804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14894c0-5960-49c5-a4ce-f8c214b6b010 admin admin] PATH_INFO: `/v3/projects/814055191dd4462ba473eddad263136c/users/af53f488e38e4c8798aa1a38b2b3136b/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:33.369963 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39dd50-6382-4ea5-8022-747584c41071 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:33.373163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14894c0-5960-49c5-a4ce-f8c214b6b010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=814055191dd4462ba473eddad263136c, user_id=af53f488e38e4c8798aa1a38b2b3136b, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:33.378553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14894c0-5960-49c5-a4ce-f8c214b6b010 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:33.398454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 938/1879] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:33.399539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39dd50-6382-4ea5-8022-747584c41071 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:33.400389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 941/1879] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:33 2022] PUT /identity/v3/projects/814055191dd4462ba473eddad263136c/users/af53f488e38e4c8798aa1a38b2b3136b/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:33.407767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b78e6e-b7da-477f-85f7-c8eab19aa1cc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:33.408173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b78e6e-b7da-477f-85f7-c8eab19aa1cc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:33.408475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b78e6e-b7da-477f-85f7-c8eab19aa1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:33.412068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39dd50-6382-4ea5-8022-747584c41071 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:33.454045 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c4b78e6e-b7da-477f-85f7-c8eab19aa1cc None None] MFA Rules not processed for user `af53f488e38e4c8798aa1a38b2b3136b`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:33.456248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c4b78e6e-b7da-477f-85f7-c8eab19aa1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:33.535320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcd83a-4878-47e6-8b9c-f9e6e2f6f0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:33.536254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcd83a-4878-47e6-8b9c-f9e6e2f6f0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:33.536540 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcd83a-4878-47e6-8b9c-f9e6e2f6f0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:33.537205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcd83a-4878-47e6-8b9c-f9e6e2f6f0c4 admin admin] PATH_INFO: `/v3/projects/7bdb01d55d2a4456bd4c226ac9679fab/users/9dc983dfd21f4a4ea30b0412a97720ed/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:33.555797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcd83a-4878-47e6-8b9c-f9e6e2f6f0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bdb01d55d2a4456bd4c226ac9679fab, user_id=9dc983dfd21f4a4ea30b0412a97720ed, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:33.559015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcd83a-4878-47e6-8b9c-f9e6e2f6f0c4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:33.576128 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 942/1880] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:33.588331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b78e6e-b7da-477f-85f7-c8eab19aa1cc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:33.590265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c4b78e6e-b7da-477f-85f7-c8eab19aa1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:33.646145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 939/1881] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:33 2022] PUT /identity/v3/projects/7bdb01d55d2a4456bd4c226ac9679fab/users/9dc983dfd21f4a4ea30b0412a97720ed/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:33.650080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcd83a-4878-47e6-8b9c-f9e6e2f6f0c4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:33.656936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b78e6e-b7da-477f-85f7-c8eab19aa1cc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:33.695400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:33.715299 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3df3f-e0ca-4506-bd01-508b8bbf0c12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:33.715941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3df3f-e0ca-4506-bd01-508b8bbf0c12 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:33.716257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3df3f-e0ca-4506-bd01-508b8bbf0c12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:33.716540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3df3f-e0ca-4506-bd01-508b8bbf0c12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:33.725571 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcd83a-4878-47e6-8b9c-f9e6e2f6f0c4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:33.730823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eda492-e103-4a0f-be8d-e4236f760d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:33.731486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eda492-e103-4a0f-be8d-e4236f760d9a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:33.731768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eda492-e103-4a0f-be8d-e4236f760d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:33.732074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eda492-e103-4a0f-be8d-e4236f760d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:33.732771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eda492-e103-4a0f-be8d-e4236f760d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:33.734842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eda492-e103-4a0f-be8d-e4236f760d9a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:33.745320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 940/1882] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:33.749866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eda492-e103-4a0f-be8d-e4236f760d9a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:33.760054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eda492-e103-4a0f-be8d-e4236f760d9a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:33.765230 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6b845-8f3e-4b91-8e0f-b5d51413992e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:33.765925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6b845-8f3e-4b91-8e0f-b5d51413992e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:33.766205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6b845-8f3e-4b91-8e0f-b5d51413992e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:33.766476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6b845-8f3e-4b91-8e0f-b5d51413992e admin admin] PATH_INFO: `/v3/projects/7bdb01d55d2a4456bd4c226ac9679fab/users/9dc983dfd21f4a4ea30b0412a97720ed/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:33.768546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6b845-8f3e-4b91-8e0f-b5d51413992e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bdb01d55d2a4456bd4c226ac9679fab, user_id=9dc983dfd21f4a4ea30b0412a97720ed, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:33.771195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6b845-8f3e-4b91-8e0f-b5d51413992e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:33.781459 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 941/1883] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:33 2022] PUT /identity/v3/projects/7bdb01d55d2a4456bd4c226ac9679fab/users/9dc983dfd21f4a4ea30b0412a97720ed/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:33.781728 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3df3f-e0ca-4506-bd01-508b8bbf0c12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:33.787268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3df3f-e0ca-4506-bd01-508b8bbf0c12 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:33.792115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89c5bd-c829-4e9d-8332-b28b62c203fc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:33.792401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89c5bd-c829-4e9d-8332-b28b62c203fc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:33.792702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89c5bd-c829-4e9d-8332-b28b62c203fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:33.793878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 943/1884] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:42:33.824465 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5c89c5bd-c829-4e9d-8332-b28b62c203fc None None] MFA Rules not processed for user `9dc983dfd21f4a4ea30b0412a97720ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:33.825759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5c89c5bd-c829-4e9d-8332-b28b62c203fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:33.883684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:33.929397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 942/1885] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:33.945614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3df3f-e0ca-4506-bd01-508b8bbf0c12 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:33.958086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89c5bd-c829-4e9d-8332-b28b62c203fc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:33.958865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3df3f-e0ca-4506-bd01-508b8bbf0c12 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:34.035843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89c5bd-c829-4e9d-8332-b28b62c203fc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:34.049601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:34.050520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:34.050804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:34.051220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] PATH_INFO: `/v3/users/fcd145d9bd644f3bb150dfc7d43d1c8a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:34.056252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcd145d9bd644f3bb150dfc7d43d1c8a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:34.057309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2a542-206c-409a-845f-2f16177db0f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:34.058034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2a542-206c-409a-845f-2f16177db0f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:34.058469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2a542-206c-409a-845f-2f16177db0f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:34.058730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:34.058866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2a542-206c-409a-845f-2f16177db0f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:34.069971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2a542-206c-409a-845f-2f16177db0f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:34.071620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2a542-206c-409a-845f-2f16177db0f8 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:34.083463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 943/1886] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:34.223598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2a542-206c-409a-845f-2f16177db0f8 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:34.225019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cfb2a542-206c-409a-845f-2f16177db0f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:34.237174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2a542-206c-409a-845f-2f16177db0f8 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:34.240921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda74ed-bbdc-4b62-8dc8-acde593592c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:34.241535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda74ed-bbdc-4b62-8dc8-acde593592c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:34.241822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda74ed-bbdc-4b62-8dc8-acde593592c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:34.242100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda74ed-bbdc-4b62-8dc8-acde593592c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:34.304165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcd145d9bd644f3bb150dfc7d43d1c8a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:34.304887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcd145d9bd644f3bb150dfc7d43d1c8a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:34.305535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcd145d9bd644f3bb150dfc7d43d1c8a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:34.311573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda74ed-bbdc-4b62-8dc8-acde593592c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:34.313352 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda74ed-bbdc-4b62-8dc8-acde593592c0 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:34.332046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcd145d9bd644f3bb150dfc7d43d1c8a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:34.332720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 944/1887] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:42:34.399643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcd145d9bd644f3bb150dfc7d43d1c8a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:34.416121 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 944/1888] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:33 2022] DELETE /identity/v3/users/fcd145d9bd644f3bb150dfc7d43d1c8a => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:34.424684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb3775-c7f9-4a94-994a-b65e0a0e8649 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:34.425099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb3775-c7f9-4a94-994a-b65e0a0e8649 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:34.426092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb3775-c7f9-4a94-994a-b65e0a0e8649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:34.445894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-86eb3775-c7f9-4a94-994a-b65e0a0e8649 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:34.447151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-86eb3775-c7f9-4a94-994a-b65e0a0e8649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:34.507940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 945/1889] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:34.518501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:34.519992 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:34.530298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:34.580613 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39874e80-d08d-42a1-b40b-dbc781d30efd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:34.584516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1edf69-88d3-42fb-bdad-1343c9f9649e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:34.585149 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1edf69-88d3-42fb-bdad-1343c9f9649e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:34.585433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1edf69-88d3-42fb-bdad-1343c9f9649e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:34.585709 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1edf69-88d3-42fb-bdad-1343c9f9649e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:34.595734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1edf69-88d3-42fb-bdad-1343c9f9649e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:34.597269 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1edf69-88d3-42fb-bdad-1343c9f9649e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:34.607072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 945/1890] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:35.118117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb3775-c7f9-4a94-994a-b65e0a0e8649 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:35.119577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-86eb3775-c7f9-4a94-994a-b65e0a0e8649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:35.133482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb3775-c7f9-4a94-994a-b65e0a0e8649 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:35.142855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:35.144309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:35.144309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:35.144309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] PATH_INFO: `/v3/projects/56d106a358cd4ac8a4286cc4a0c41c8e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:35.147032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] RBAC: Authorizing `identity:delete_project(project_id=56d106a358cd4ac8a4286cc4a0c41c8e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:35.149795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:35.163440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:35.256068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56d106a358cd4ac8a4286cc4a0c41c8e', 'request_id': 'req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:35.343644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56d106a358cd4ac8a4286cc4a0c41c8e', 'request_id': 'req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:35.361616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56d106a358cd4ac8a4286cc4a0c41c8e', 'request_id': 'req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:35.375585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] The token cache is being invalidate because project 56d106a358cd4ac8a4286cc4a0c41c8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:35.375873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:35.376842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:35.381507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 946/1891] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:35 2022] DELETE /identity/v3/projects/56d106a358cd4ac8a4286cc4a0c41c8e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:35.382700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1edf69-88d3-42fb-bdad-1343c9f9649e service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:35.395513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1edf69-88d3-42fb-bdad-1343c9f9649e service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:35.454402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4a251-863b-46a7-b930-e4cbe36ac3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:35.455083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a251-863b-46a7-b930-e4cbe36ac3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:35.455372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a251-863b-46a7-b930-e4cbe36ac3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:35.455715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a251-863b-46a7-b930-e4cbe36ac3a7 admin admin] PATH_INFO: `/v3/users/af53f488e38e4c8798aa1a38b2b3136b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:35.456939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4a251-863b-46a7-b930-e4cbe36ac3a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af53f488e38e4c8798aa1a38b2b3136b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:35.459247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4a251-863b-46a7-b930-e4cbe36ac3a7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:35.635347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ece4a251-863b-46a7-b930-e4cbe36ac3a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af53f488e38e4c8798aa1a38b2b3136b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:35.636054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ece4a251-863b-46a7-b930-e4cbe36ac3a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af53f488e38e4c8798aa1a38b2b3136b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:35.636667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ece4a251-863b-46a7-b930-e4cbe36ac3a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af53f488e38e4c8798aa1a38b2b3136b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:35.649616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ece4a251-863b-46a7-b930-e4cbe36ac3a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af53f488e38e4c8798aa1a38b2b3136b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:35.748565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ece4a251-863b-46a7-b930-e4cbe36ac3a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af53f488e38e4c8798aa1a38b2b3136b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:35.765154 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 946/1892] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:35 2022] DELETE /identity/v3/users/af53f488e38e4c8798aa1a38b2b3136b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:36.316205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:36.317530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:36.333775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8c9a6-f7d9-487a-b188-d3f27a3ef0db admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:36.414936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:36.415727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:36.416042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:36.416323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] PATH_INFO: `/v3/projects/814055191dd4462ba473eddad263136c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:36.417511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=814055191dd4462ba473eddad263136c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:36.419495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:36.549303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '814055191dd4462ba473eddad263136c', 'request_id': 'req-7abad957-4f2b-4603-bdf1-4391f6c2e69a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:36.622477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '814055191dd4462ba473eddad263136c', 'request_id': 'req-7abad957-4f2b-4603-bdf1-4391f6c2e69a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:36.629530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '814055191dd4462ba473eddad263136c', 'request_id': 'req-7abad957-4f2b-4603-bdf1-4391f6c2e69a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:36.641209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] The token cache is being invalidate because project 814055191dd4462ba473eddad263136c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:36.641500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:36.642151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7abad957-4f2b-4603-bdf1-4391f6c2e69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:36.644399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 947/1893] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:36 2022] DELETE /identity/v3/projects/814055191dd4462ba473eddad263136c => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:36.649064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620a23-ea3b-48dc-ac0e-6e0a6917c236 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:36.649564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620a23-ea3b-48dc-ac0e-6e0a6917c236 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:36.649843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620a23-ea3b-48dc-ac0e-6e0a6917c236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:36.671746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c3620a23-ea3b-48dc-ac0e-6e0a6917c236 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:36.673081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c3620a23-ea3b-48dc-ac0e-6e0a6917c236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:36.699801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:36.736421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 947/1894] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:36.744208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9903a2-af71-4f13-8a69-a976b2c350e3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:36.744502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9903a2-af71-4f13-8a69-a976b2c350e3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:36.744777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9903a2-af71-4f13-8a69-a976b2c350e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:36.774681 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3c9903a2-af71-4f13-8a69-a976b2c350e3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:36.775944 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3c9903a2-af71-4f13-8a69-a976b2c350e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:36.786964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 948/1895] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:36.791228 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3620a23-ea3b-48dc-ac0e-6e0a6917c236 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:36.801582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3620a23-ea3b-48dc-ac0e-6e0a6917c236 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:36.806246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb04936-b819-4a2b-bbe6-653cc33b35c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:36.806859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04936-b819-4a2b-bbe6-653cc33b35c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:36.807141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04936-b819-4a2b-bbe6-653cc33b35c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:36.807421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04936-b819-4a2b-bbe6-653cc33b35c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:36.808213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb04936-b819-4a2b-bbe6-653cc33b35c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:36.808520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb04936-b819-4a2b-bbe6-653cc33b35c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:36.810165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb04936-b819-4a2b-bbe6-653cc33b35c3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:36.823391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 948/1896] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:42:36.832740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9903a2-af71-4f13-8a69-a976b2c350e3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:36.842246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9903a2-af71-4f13-8a69-a976b2c350e3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:36.847862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434967f5-17bd-462f-8001-5d00583ec13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:36.848495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434967f5-17bd-462f-8001-5d00583ec13c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:36.848778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434967f5-17bd-462f-8001-5d00583ec13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:36.853982 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434967f5-17bd-462f-8001-5d00583ec13c admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:36.853982 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434967f5-17bd-462f-8001-5d00583ec13c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:36.853982 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434967f5-17bd-462f-8001-5d00583ec13c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:36.957501 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:36.966508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb04936-b819-4a2b-bbe6-653cc33b35c3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:36.972067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 949/1897] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:36 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:36.984124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb04936-b819-4a2b-bbe6-653cc33b35c3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:37.053803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2893f0-698c-4a6a-9618-12e049c3b0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:37.054436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2893f0-698c-4a6a-9618-12e049c3b0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:37.054772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2893f0-698c-4a6a-9618-12e049c3b0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:37.055057 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2893f0-698c-4a6a-9618-12e049c3b0d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:37.055913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2893f0-698c-4a6a-9618-12e049c3b0d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:37.058085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2893f0-698c-4a6a-9618-12e049c3b0d9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:37.183779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 949/1898] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:36 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:37.188515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434967f5-17bd-462f-8001-5d00583ec13c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:37.198781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434967f5-17bd-462f-8001-5d00583ec13c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:37.206107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252fadd-4869-431f-a2b0-c7471922b8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:37.206107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252fadd-4869-431f-a2b0-c7471922b8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:37.206107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252fadd-4869-431f-a2b0-c7471922b8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:37.206107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252fadd-4869-431f-a2b0-c7471922b8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:37.206436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252fadd-4869-431f-a2b0-c7471922b8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:37.207784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252fadd-4869-431f-a2b0-c7471922b8a6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:37.218365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 950/1899] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:37.222703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2893f0-698c-4a6a-9618-12e049c3b0d9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:37.232870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2893f0-698c-4a6a-9618-12e049c3b0d9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:37.237571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7227c-4fdf-4676-bc71-d785f1ac7add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:37.238280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7227c-4fdf-4676-bc71-d785f1ac7add admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:37.238503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7227c-4fdf-4676-bc71-d785f1ac7add admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:37.238786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7227c-4fdf-4676-bc71-d785f1ac7add admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:37.239517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7227c-4fdf-4676-bc71-d785f1ac7add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:37.241163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7227c-4fdf-4676-bc71-d785f1ac7add admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:37.251766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 950/1900] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:37.256198 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252fadd-4869-431f-a2b0-c7471922b8a6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:37.267337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252fadd-4869-431f-a2b0-c7471922b8a6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:37.271893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af3e27-5d64-435a-94bf-a9e03e4b2099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:37.273023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af3e27-5d64-435a-94bf-a9e03e4b2099 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:37.273023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af3e27-5d64-435a-94bf-a9e03e4b2099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:37.273258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af3e27-5d64-435a-94bf-a9e03e4b2099 admin admin] PATH_INFO: `/v3/projects/31d837bb62fc4fd1bba901d967a59755/users/add9b4be88f14169a27355280354c5ef/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:37.288112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af3e27-5d64-435a-94bf-a9e03e4b2099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d837bb62fc4fd1bba901d967a59755, user_id=add9b4be88f14169a27355280354c5ef, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:37.290941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af3e27-5d64-435a-94bf-a9e03e4b2099 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:37.371284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 951/1901] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:37 2022] PUT /identity/v3/projects/31d837bb62fc4fd1bba901d967a59755/users/add9b4be88f14169a27355280354c5ef/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:37.382247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca13ca-349d-416b-bfbd-2f286ce1fe71 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:37.382538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca13ca-349d-416b-bfbd-2f286ce1fe71 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:37.382815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca13ca-349d-416b-bfbd-2f286ce1fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:37.413082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-08ca13ca-349d-416b-bfbd-2f286ce1fe71 None None] MFA Rules not processed for user `add9b4be88f14169a27355280354c5ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:37.414323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-08ca13ca-349d-416b-bfbd-2f286ce1fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:37.479834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 951/1902] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:37.484686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af3e27-5d64-435a-94bf-a9e03e4b2099 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:37.544889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af3e27-5d64-435a-94bf-a9e03e4b2099 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:37.549869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cfaba-97fb-4335-89ce-7a246a5f4118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:37.550487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cfaba-97fb-4335-89ce-7a246a5f4118 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:37.550774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cfaba-97fb-4335-89ce-7a246a5f4118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:37.551052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cfaba-97fb-4335-89ce-7a246a5f4118 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:37.551734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cfaba-97fb-4335-89ce-7a246a5f4118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:37.553669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cfaba-97fb-4335-89ce-7a246a5f4118 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:37.662927 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 952/1903] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:37 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:37.666846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca13ca-349d-416b-bfbd-2f286ce1fe71 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:37.678225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca13ca-349d-416b-bfbd-2f286ce1fe71 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:37.698935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:37.738630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f58d15-a9ff-4243-a135-3149d565f669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:37.739062 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f58d15-a9ff-4243-a135-3149d565f669 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:37.739407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f58d15-a9ff-4243-a135-3149d565f669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:37.739770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f58d15-a9ff-4243-a135-3149d565f669 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:37.740493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f58d15-a9ff-4243-a135-3149d565f669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:37.742470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f58d15-a9ff-4243-a135-3149d565f669 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:37.875068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 952/1904] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:37 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:37.879176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cfaba-97fb-4335-89ce-7a246a5f4118 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:37.889608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cfaba-97fb-4335-89ce-7a246a5f4118 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:37.894608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017995e-3a69-4c44-99c6-dd69cb438876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:37.895224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017995e-3a69-4c44-99c6-dd69cb438876 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:37.895501 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017995e-3a69-4c44-99c6-dd69cb438876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:37.900034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017995e-3a69-4c44-99c6-dd69cb438876 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:37.900034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017995e-3a69-4c44-99c6-dd69cb438876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:37.900034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017995e-3a69-4c44-99c6-dd69cb438876 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:37.909412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 953/1905] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:37.914000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f58d15-a9ff-4243-a135-3149d565f669 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:37.926749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f58d15-a9ff-4243-a135-3149d565f669 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:37.947854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161678f3-78c7-44b4-8464-2f38e1651490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:37.948889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161678f3-78c7-44b4-8464-2f38e1651490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:37.952382 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161678f3-78c7-44b4-8464-2f38e1651490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:37.952895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161678f3-78c7-44b4-8464-2f38e1651490 admin admin] PATH_INFO: `/v3/projects/437834117aaa451c8b1a1815321cf814/users/0198e27eb7324cd9b2718b63e340bdb9/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:37.981681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161678f3-78c7-44b4-8464-2f38e1651490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=437834117aaa451c8b1a1815321cf814, user_id=0198e27eb7324cd9b2718b63e340bdb9, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:37.985190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161678f3-78c7-44b4-8464-2f38e1651490 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:38.095265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 953/1906] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:37 2022] PUT /identity/v3/projects/437834117aaa451c8b1a1815321cf814/users/0198e27eb7324cd9b2718b63e340bdb9/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:38.096304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017995e-3a69-4c44-99c6-dd69cb438876 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:38.106535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017995e-3a69-4c44-99c6-dd69cb438876 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:38.196219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccaf1d-bb97-4d62-840d-32334882aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:38.200192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccaf1d-bb97-4d62-840d-32334882aae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:38.200192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccaf1d-bb97-4d62-840d-32334882aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:38.200192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccaf1d-bb97-4d62-840d-32334882aae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:38.200192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccaf1d-bb97-4d62-840d-32334882aae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:38.200702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccaf1d-bb97-4d62-840d-32334882aae5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:38.229255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 954/1907] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:38.230414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161678f3-78c7-44b4-8464-2f38e1651490 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:38.246081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161678f3-78c7-44b4-8464-2f38e1651490 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:38.252790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e4c0c-838e-4d6a-b3c4-aaa363318905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:38.254008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e4c0c-838e-4d6a-b3c4-aaa363318905 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:38.254530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e4c0c-838e-4d6a-b3c4-aaa363318905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:38.255076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e4c0c-838e-4d6a-b3c4-aaa363318905 admin admin] PATH_INFO: `/v3/projects/437834117aaa451c8b1a1815321cf814/users/0198e27eb7324cd9b2718b63e340bdb9/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:38.257409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e4c0c-838e-4d6a-b3c4-aaa363318905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=437834117aaa451c8b1a1815321cf814, user_id=0198e27eb7324cd9b2718b63e340bdb9, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:38.260798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e4c0c-838e-4d6a-b3c4-aaa363318905 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:38.276270 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 954/1908] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:38 2022] PUT /identity/v3/projects/437834117aaa451c8b1a1815321cf814/users/0198e27eb7324cd9b2718b63e340bdb9/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:38.289413 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c8761-0acb-4914-a43c-d480bc464e2f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:38.289716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c8761-0acb-4914-a43c-d480bc464e2f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:38.289998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c8761-0acb-4914-a43c-d480bc464e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:38.322033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-687c8761-0acb-4914-a43c-d480bc464e2f None None] MFA Rules not processed for user `0198e27eb7324cd9b2718b63e340bdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:38.323266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-687c8761-0acb-4914-a43c-d480bc464e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:38.363821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:38.440443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 955/1909] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:38.494076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e4c0c-838e-4d6a-b3c4-aaa363318905 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:38.495830 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-be0e4c0c-838e-4d6a-b3c4-aaa363318905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:38.557561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e4c0c-838e-4d6a-b3c4-aaa363318905 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:38.562333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15b3fd-c965-4974-9981-1bc98d028fca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:38.562947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15b3fd-c965-4974-9981-1bc98d028fca service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:38.563233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15b3fd-c965-4974-9981-1bc98d028fca service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:38.563508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15b3fd-c965-4974-9981-1bc98d028fca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:38.573034 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15b3fd-c965-4974-9981-1bc98d028fca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:38.574530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15b3fd-c965-4974-9981-1bc98d028fca service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:38.585230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 955/1910] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:39.469513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c8761-0acb-4914-a43c-d480bc464e2f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:39.470980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-687c8761-0acb-4914-a43c-d480bc464e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:39.559206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c8761-0acb-4914-a43c-d480bc464e2f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:39.563907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe82f35-d543-4a23-9ba3-1b9a6457d554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:39.564583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe82f35-d543-4a23-9ba3-1b9a6457d554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:39.564865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe82f35-d543-4a23-9ba3-1b9a6457d554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:39.565147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe82f35-d543-4a23-9ba3-1b9a6457d554 admin admin] PATH_INFO: `/v3/users/55a0d313bb9e4648a53548168614e440` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:39.566346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe82f35-d543-4a23-9ba3-1b9a6457d554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a0d313bb9e4648a53548168614e440)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:39.568388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe82f35-d543-4a23-9ba3-1b9a6457d554 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:39.672689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15b3fd-c965-4974-9981-1bc98d028fca service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:39.685878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15b3fd-c965-4974-9981-1bc98d028fca service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:39.694015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c095db-b19b-4081-85df-71c92f4b2b1d tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0198e27eb7324cd9b2718b63e340bdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '437834117aaa451c8b1a1815321cf814', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:39.695132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c095db-b19b-4081-85df-71c92f4b2b1d tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:39.695588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c095db-b19b-4081-85df-71c92f4b2b1d tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:39.696077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c095db-b19b-4081-85df-71c92f4b2b1d tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:39.697565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c095db-b19b-4081-85df-71c92f4b2b1d tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:39.698281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c095db-b19b-4081-85df-71c92f4b2b1d tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:39.703053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c095db-b19b-4081-85df-71c92f4b2b1d tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:39.720168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 956/1911] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:42:39.721264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c095db-b19b-4081-85df-71c92f4b2b1d tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:39.732150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c095db-b19b-4081-85df-71c92f4b2b1d tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:39.737861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dd624-8e21-43a2-868b-f2aff210697a tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0198e27eb7324cd9b2718b63e340bdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '437834117aaa451c8b1a1815321cf814', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:39.744760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dd624-8e21-43a2-868b-f2aff210697a tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:39.744760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dd624-8e21-43a2-868b-f2aff210697a tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:39.744760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dd624-8e21-43a2-868b-f2aff210697a tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:39.744760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37dd624-8e21-43a2-868b-f2aff210697a tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:39.744760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37dd624-8e21-43a2-868b-f2aff210697a tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:39.779528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2fe82f35-d543-4a23-9ba3-1b9a6457d554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a0d313bb9e4648a53548168614e440'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:39.780925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2fe82f35-d543-4a23-9ba3-1b9a6457d554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a0d313bb9e4648a53548168614e440'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:39.789590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2fe82f35-d543-4a23-9ba3-1b9a6457d554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a0d313bb9e4648a53548168614e440'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:39.808874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2fe82f35-d543-4a23-9ba3-1b9a6457d554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a0d313bb9e4648a53548168614e440'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:39.891603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 957/1912] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:39 2022] POST /identity/v3/projects => generated 354 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:39.895899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dd624-8e21-43a2-868b-f2aff210697a tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:39.897651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d37dd624-8e21-43a2-868b-f2aff210697a tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:39.946625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2fe82f35-d543-4a23-9ba3-1b9a6457d554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a0d313bb9e4648a53548168614e440'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:39.955827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 956/1913] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:39 2022] DELETE /identity/v3/users/55a0d313bb9e4648a53548168614e440 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:39.960758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ed609-bb6e-45e5-b418-d17838a5fa2d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:39.961106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ed609-bb6e-45e5-b418-d17838a5fa2d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:39.961395 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ed609-bb6e-45e5-b418-d17838a5fa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:39.965597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:39.985524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-965ed609-bb6e-45e5-b418-d17838a5fa2d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:39.987169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dd624-8e21-43a2-868b-f2aff210697a tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:39.987621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-965ed609-bb6e-45e5-b418-d17838a5fa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:40.022874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:40.071719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 957/1914] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:40.080073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee0f61-957d-4c9c-a2c6-bbfddc9dc66b tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0198e27eb7324cd9b2718b63e340bdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '437834117aaa451c8b1a1815321cf814', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:40.080916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee0f61-957d-4c9c-a2c6-bbfddc9dc66b tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:40.081051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee0f61-957d-4c9c-a2c6-bbfddc9dc66b tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:40.081257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee0f61-957d-4c9c-a2c6-bbfddc9dc66b tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:40.082368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee0f61-957d-4c9c-a2c6-bbfddc9dc66b tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:40.082685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee0f61-957d-4c9c-a2c6-bbfddc9dc66b tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:40.084386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ed609-bb6e-45e5-b418-d17838a5fa2d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:40.084626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee0f61-957d-4c9c-a2c6-bbfddc9dc66b tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:40.085791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-965ed609-bb6e-45e5-b418-d17838a5fa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:40.100192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 958/1915] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:42:40.142829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee0f61-957d-4c9c-a2c6-bbfddc9dc66b tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:40.175160 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee0f61-957d-4c9c-a2c6-bbfddc9dc66b tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:40.189847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0198e27eb7324cd9b2718b63e340bdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '437834117aaa451c8b1a1815321cf814', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:40.190618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:40.190796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:40.195407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] PATH_INFO: `/v3/projects/86a87c2651e54307be8de35dce3bb935` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:40.195407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: Authorizing `identity:delete_project(project_id=86a87c2651e54307be8de35dce3bb935)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:40.195719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ed609-bb6e-45e5-b418-d17838a5fa2d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:40.195811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:40.198128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79918202-d936-4b9a-9ca1-09b6ab77e7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:40.199259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79918202-d936-4b9a-9ca1-09b6ab77e7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:40.199830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79918202-d936-4b9a-9ca1-09b6ab77e7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:40.200338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79918202-d936-4b9a-9ca1-09b6ab77e7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:40.214191 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79918202-d936-4b9a-9ca1-09b6ab77e7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:40.216031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79918202-d936-4b9a-9ca1-09b6ab77e7c1 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:40.237193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 958/1916] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:40.365961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86a87c2651e54307be8de35dce3bb935', 'request_id': 'req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:40.376516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86a87c2651e54307be8de35dce3bb935', 'request_id': 'req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:40.446896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86a87c2651e54307be8de35dce3bb935', 'request_id': 'req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:40.464899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] The token cache is being invalidate because project 86a87c2651e54307be8de35dce3bb935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:40.465181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:40.465879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:40.470448 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 959/1917] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:40 2022] DELETE /identity/v3/projects/86a87c2651e54307be8de35dce3bb935 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:40.982076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79918202-d936-4b9a-9ca1-09b6ab77e7c1 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:40.983671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-79918202-d936-4b9a-9ca1-09b6ab77e7c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:40.999032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79918202-d936-4b9a-9ca1-09b6ab77e7c1 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:41.073162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:41.073928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:41.074641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:41.074753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] PATH_INFO: `/v3/projects/851f8f8a6e464673a5cf2b0c047bb1a0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:41.076445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] RBAC: Authorizing `identity:delete_project(project_id=851f8f8a6e464673a5cf2b0c047bb1a0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:41.078788 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:41.210093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '851f8f8a6e464673a5cf2b0c047bb1a0', 'request_id': 'req-5b72647c-9cc2-4962-bfe0-8d575350d44e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.313000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '851f8f8a6e464673a5cf2b0c047bb1a0', 'request_id': 'req-5b72647c-9cc2-4962-bfe0-8d575350d44e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.321432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '851f8f8a6e464673a5cf2b0c047bb1a0', 'request_id': 'req-5b72647c-9cc2-4962-bfe0-8d575350d44e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.325641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:41.328165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:41.337656 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] The token cache is being invalidate because project 851f8f8a6e464673a5cf2b0c047bb1a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:41.338296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.339018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.344068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 959/1918] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:40 2022] DELETE /identity/v3/projects/851f8f8a6e464673a5cf2b0c047bb1a0 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:41.344483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1e685-fceb-4aa9-a6cc-8dd3b19b271c tempest-VolumeQuotaClassesTest-998626555 tempest-VolumeQuotaClassesTest-998626555-project-admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:41.345268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:41.346933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:41.367002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:41.432465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60304967-c027-4166-9995-70988bfa9088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:41.433262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60304967-c027-4166-9995-70988bfa9088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:41.433772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60304967-c027-4166-9995-70988bfa9088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:41.434102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60304967-c027-4166-9995-70988bfa9088 admin admin] PATH_INFO: `/v3/users/add9b4be88f14169a27355280354c5ef` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:41.435590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60304967-c027-4166-9995-70988bfa9088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=add9b4be88f14169a27355280354c5ef)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:41.438509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60304967-c027-4166-9995-70988bfa9088 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:41.439811 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72647c-9cc2-4962-bfe0-8d575350d44e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:41.447585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:41.448282 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:41.448706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:41.455238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] PATH_INFO: `/v3/users/9dc983dfd21f4a4ea30b0412a97720ed` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:41.455238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc983dfd21f4a4ea30b0412a97720ed)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:41.455462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:41.455555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:41.658098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-60304967-c027-4166-9995-70988bfa9088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add9b4be88f14169a27355280354c5ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.658985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-60304967-c027-4166-9995-70988bfa9088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add9b4be88f14169a27355280354c5ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.659602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-60304967-c027-4166-9995-70988bfa9088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add9b4be88f14169a27355280354c5ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.674087 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-60304967-c027-4166-9995-70988bfa9088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add9b4be88f14169a27355280354c5ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.688417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc983dfd21f4a4ea30b0412a97720ed'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.689150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc983dfd21f4a4ea30b0412a97720ed'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.696264 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc983dfd21f4a4ea30b0412a97720ed'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.708358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc983dfd21f4a4ea30b0412a97720ed'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.746217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-60304967-c027-4166-9995-70988bfa9088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add9b4be88f14169a27355280354c5ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.760298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 960/1919] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:41 2022] DELETE /identity/v3/users/add9b4be88f14169a27355280354c5ef => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:41.764803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d57b3-9039-484f-bfb6-b0a6c346b873 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:41.765169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d57b3-9039-484f-bfb6-b0a6c346b873 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:41.765519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d57b3-9039-484f-bfb6-b0a6c346b873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:41.790364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5e1d57b3-9039-484f-bfb6-b0a6c346b873 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:41.792144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5e1d57b3-9039-484f-bfb6-b0a6c346b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:41.867896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 961/1920] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:41.868815 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc983dfd21f4a4ea30b0412a97720ed'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:41.877598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 960/1921] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:41 2022] DELETE /identity/v3/users/9dc983dfd21f4a4ea30b0412a97720ed => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:41.909813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d57b3-9039-484f-bfb6-b0a6c346b873 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:41.912502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5e1d57b3-9039-484f-bfb6-b0a6c346b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:41.998007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d57b3-9039-484f-bfb6-b0a6c346b873 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:42.091289 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aea5f7-b307-4403-996b-1cefc4ac83cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:42.091957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aea5f7-b307-4403-996b-1cefc4ac83cf service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:42.092286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aea5f7-b307-4403-996b-1cefc4ac83cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:42.092805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aea5f7-b307-4403-996b-1cefc4ac83cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:42.163175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aea5f7-b307-4403-996b-1cefc4ac83cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:42.164807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aea5f7-b307-4403-996b-1cefc4ac83cf service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:42.178056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 962/1922] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:42.618093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:42.619617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:42.634943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47429f81-3688-43a4-842b-1f771ea1fdb2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:42.641090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:42.643096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:42.643096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:42.643096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] PATH_INFO: `/v3/projects/7bdb01d55d2a4456bd4c226ac9679fab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:42.644148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bdb01d55d2a4456bd4c226ac9679fab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:42.646077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:42.722015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aea5f7-b307-4403-996b-1cefc4ac83cf service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:42.722809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a3aea5f7-b307-4403-996b-1cefc4ac83cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:42.755745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aea5f7-b307-4403-996b-1cefc4ac83cf service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:42.761704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:42.762559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:42.762859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:42.763283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] PATH_INFO: `/v3/projects/31d837bb62fc4fd1bba901d967a59755` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:42.766394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31d837bb62fc4fd1bba901d967a59755)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:42.768925 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:42.784915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bdb01d55d2a4456bd4c226ac9679fab', 'request_id': 'req-93d0bc03-33ff-4739-a5a2-77a850e6688d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:42.850971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bdb01d55d2a4456bd4c226ac9679fab', 'request_id': 'req-93d0bc03-33ff-4739-a5a2-77a850e6688d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:42.860023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bdb01d55d2a4456bd4c226ac9679fab', 'request_id': 'req-93d0bc03-33ff-4739-a5a2-77a850e6688d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:42.875277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:42.880625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] The token cache is being invalidate because project 7bdb01d55d2a4456bd4c226ac9679fab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:42.884117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:42.884117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-93d0bc03-33ff-4739-a5a2-77a850e6688d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:42.884117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 961/1923] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:42 2022] DELETE /identity/v3/projects/7bdb01d55d2a4456bd4c226ac9679fab => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:42.888378 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2cea6-852e-4320-9411-0db5248a161b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:42.888671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2cea6-852e-4320-9411-0db5248a161b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:42.888951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2cea6-852e-4320-9411-0db5248a161b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:42.899429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d837bb62fc4fd1bba901d967a59755', 'request_id': 'req-a8c94bce-7aca-46c7-a474-a297fea66862'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:42.906876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d837bb62fc4fd1bba901d967a59755', 'request_id': 'req-a8c94bce-7aca-46c7-a474-a297fea66862'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:42.910005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-89b2cea6-852e-4320-9411-0db5248a161b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:42.911236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-89b2cea6-852e-4320-9411-0db5248a161b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:42.921739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:42.962669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d837bb62fc4fd1bba901d967a59755', 'request_id': 'req-a8c94bce-7aca-46c7-a474-a297fea66862'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:42.987668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] The token cache is being invalidate because project 31d837bb62fc4fd1bba901d967a59755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:42.988401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:42.989331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:42.990068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 962/1924] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:42.992182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 963/1925] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:42 2022] DELETE /identity/v3/projects/31d837bb62fc4fd1bba901d967a59755 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:43.000426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:43.001504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56562048-18e7-46b3-a25f-5bf967303005 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:43.001793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56562048-18e7-46b3-a25f-5bf967303005 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:43.002074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56562048-18e7-46b3-a25f-5bf967303005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:43.002619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:43.017195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c94bce-7aca-46c7-a474-a297fea66862 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:43.022991 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:43.024809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:43.024809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:43.024809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] PATH_INFO: `/v3/users/0198e27eb7324cd9b2718b63e340bdb9` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:43.025126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-56562048-18e7-46b3-a25f-5bf967303005 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:43.026042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=0198e27eb7324cd9b2718b63e340bdb9)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:43.026162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-56562048-18e7-46b3-a25f-5bf967303005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:43.028768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:43.037645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 963/1926] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:43.041630 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56562048-18e7-46b3-a25f-5bf967303005 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:43.058397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56562048-18e7-46b3-a25f-5bf967303005 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:43.065296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64933a07-da40-4a83-b68a-f3069f20a9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:43.066056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64933a07-da40-4a83-b68a-f3069f20a9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:43.066451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64933a07-da40-4a83-b68a-f3069f20a9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:43.066642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64933a07-da40-4a83-b68a-f3069f20a9b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:43.067501 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64933a07-da40-4a83-b68a-f3069f20a9b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:43.067809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64933a07-da40-4a83-b68a-f3069f20a9b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:43.069866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64933a07-da40-4a83-b68a-f3069f20a9b4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:43.083163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 964/1927] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:42:43.087528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64933a07-da40-4a83-b68a-f3069f20a9b4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:43.098637 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64933a07-da40-4a83-b68a-f3069f20a9b4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:43.104252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863205b-7a21-4ba1-98da-ea93ee40fade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:43.104961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863205b-7a21-4ba1-98da-ea93ee40fade admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:43.105341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863205b-7a21-4ba1-98da-ea93ee40fade admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:43.105734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863205b-7a21-4ba1-98da-ea93ee40fade admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:43.106467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863205b-7a21-4ba1-98da-ea93ee40fade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:43.109584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863205b-7a21-4ba1-98da-ea93ee40fade admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:43.193196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 965/1928] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:43 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:43.194624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863205b-7a21-4ba1-98da-ea93ee40fade admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:43.227227 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0198e27eb7324cd9b2718b63e340bdb9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:43.228054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0198e27eb7324cd9b2718b63e340bdb9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:43.229104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0198e27eb7324cd9b2718b63e340bdb9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:43.249241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0198e27eb7324cd9b2718b63e340bdb9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:43.269995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863205b-7a21-4ba1-98da-ea93ee40fade admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:43.275099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e36e3-41dc-4643-90a2-9704a03766db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:43.275885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e36e3-41dc-4643-90a2-9704a03766db admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:43.276261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e36e3-41dc-4643-90a2-9704a03766db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:43.276540 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e36e3-41dc-4643-90a2-9704a03766db admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:43.277209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e36e3-41dc-4643-90a2-9704a03766db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:43.279235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e36e3-41dc-4643-90a2-9704a03766db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:43.313410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0198e27eb7324cd9b2718b63e340bdb9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:43.326099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 964/1929] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:42 2022] DELETE /identity/v3/users/0198e27eb7324cd9b2718b63e340bdb9 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:43.369692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 966/1930] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:43 2022] POST /identity/v3/users => generated 283 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:43.373707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:43.375291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:43.503888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def27a18-eb82-49ee-b3b8-81d47d426eda admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:43.509290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8251b1-4c4b-454a-8d51-6f5c4d0d5beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:43.509947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8251b1-4c4b-454a-8d51-6f5c4d0d5beb admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:43.510218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8251b1-4c4b-454a-8d51-6f5c4d0d5beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:43.510492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8251b1-4c4b-454a-8d51-6f5c4d0d5beb admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:43.511205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8251b1-4c4b-454a-8d51-6f5c4d0d5beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:43.513018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8251b1-4c4b-454a-8d51-6f5c4d0d5beb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:43.528026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 965/1931] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:43.529963 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e36e3-41dc-4643-90a2-9704a03766db admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:43.540697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e36e3-41dc-4643-90a2-9704a03766db admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:43.546000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10c090-6952-44eb-8bcd-ae504fac0bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:43.548556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c090-6952-44eb-8bcd-ae504fac0bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:43.548556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c090-6952-44eb-8bcd-ae504fac0bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:43.548556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c090-6952-44eb-8bcd-ae504fac0bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:43.548556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10c090-6952-44eb-8bcd-ae504fac0bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:43.549721 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10c090-6952-44eb-8bcd-ae504fac0bbf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:43.560992 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 967/1932] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:43.566399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8251b1-4c4b-454a-8d51-6f5c4d0d5beb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:43.585081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8251b1-4c4b-454a-8d51-6f5c4d0d5beb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:43.594449 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed86a9-e1e6-4532-81b2-12a98e4a6599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:43.595704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed86a9-e1e6-4532-81b2-12a98e4a6599 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:43.596193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed86a9-e1e6-4532-81b2-12a98e4a6599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:43.596580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed86a9-e1e6-4532-81b2-12a98e4a6599 admin admin] PATH_INFO: `/v3/projects/be5b129c51a349b8aa872f04e5313e87/users/dac84df2824b444eb2cfbdd6a576f24d/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:43.647127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed86a9-e1e6-4532-81b2-12a98e4a6599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be5b129c51a349b8aa872f04e5313e87, user_id=dac84df2824b444eb2cfbdd6a576f24d, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:43.650121 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed86a9-e1e6-4532-81b2-12a98e4a6599 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:43.755194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e16095-83b0-4395-ab3d-3cc619dc065b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:43.755749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e16095-83b0-4395-ab3d-3cc619dc065b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:43.755865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e16095-83b0-4395-ab3d-3cc619dc065b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:43.758550 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 966/1933] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:43 2022] PUT /identity/v3/projects/be5b129c51a349b8aa872f04e5313e87/users/dac84df2824b444eb2cfbdd6a576f24d/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:43.787349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-19e16095-83b0-4395-ab3d-3cc619dc065b None None] MFA Rules not processed for user `dac84df2824b444eb2cfbdd6a576f24d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:43.788590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-19e16095-83b0-4395-ab3d-3cc619dc065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:43.856607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 968/1934] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:43.863218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed86a9-e1e6-4532-81b2-12a98e4a6599 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:43.863218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2eed86a9-e1e6-4532-81b2-12a98e4a6599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:43.866325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e16095-83b0-4395-ab3d-3cc619dc065b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:43.931477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:43.955483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed86a9-e1e6-4532-81b2-12a98e4a6599 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:43.960771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:43.961458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e16095-83b0-4395-ab3d-3cc619dc065b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:43.961569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:43.961831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:43.962113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] PATH_INFO: `/v3/projects/437834117aaa451c8b1a1815321cf814` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:43.963283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] RBAC: Authorizing `identity:delete_project(project_id=437834117aaa451c8b1a1815321cf814)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:43.965214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:43.966680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f7888-994c-42fd-8ad3-16c45bc06aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:43.967336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f7888-994c-42fd-8ad3-16c45bc06aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:43.967678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f7888-994c-42fd-8ad3-16c45bc06aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:43.967967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f7888-994c-42fd-8ad3-16c45bc06aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:43.968961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f7888-994c-42fd-8ad3-16c45bc06aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:43.970945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f7888-994c-42fd-8ad3-16c45bc06aa2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:43.990787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:44.072387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 969/1935] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:43 2022] POST /identity/v3/projects => generated 365 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:44.076132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f7888-994c-42fd-8ad3-16c45bc06aa2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.106113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437834117aaa451c8b1a1815321cf814', 'request_id': 'req-2d437471-dab7-43bd-b840-8c34ae176ecd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:44.114440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437834117aaa451c8b1a1815321cf814', 'request_id': 'req-2d437471-dab7-43bd-b840-8c34ae176ecd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:44.171654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437834117aaa451c8b1a1815321cf814', 'request_id': 'req-2d437471-dab7-43bd-b840-8c34ae176ecd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:44.177381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f7888-994c-42fd-8ad3-16c45bc06aa2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.194876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] The token cache is being invalidate because project 437834117aaa451c8b1a1815321cf814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:44.195625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:44.196449 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d437471-dab7-43bd-b840-8c34ae176ecd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:44.199678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 967/1936] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:43 2022] DELETE /identity/v3/projects/437834117aaa451c8b1a1815321cf814 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:44.204116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8517d-a936-4494-ac28-7aa424b8b53e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.204607 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8517d-a936-4494-ac28-7aa424b8b53e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.205714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8517d-a936-4494-ac28-7aa424b8b53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.229795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cea8517d-a936-4494-ac28-7aa424b8b53e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:44.231051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cea8517d-a936-4494-ac28-7aa424b8b53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:44.250454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254dea5-63b7-47b0-ade0-c4656938c803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.251081 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254dea5-63b7-47b0-ade0-c4656938c803 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.251533 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254dea5-63b7-47b0-ade0-c4656938c803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.251813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254dea5-63b7-47b0-ade0-c4656938c803 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.252546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254dea5-63b7-47b0-ade0-c4656938c803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.254681 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254dea5-63b7-47b0-ade0-c4656938c803 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.297877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 968/1937] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:44.305584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed12800-6f25-4fcd-a6d3-6340af9dae1b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.305871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed12800-6f25-4fcd-a6d3-6340af9dae1b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.306145 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed12800-6f25-4fcd-a6d3-6340af9dae1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.324724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5ed12800-6f25-4fcd-a6d3-6340af9dae1b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:44.325914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5ed12800-6f25-4fcd-a6d3-6340af9dae1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:44.336122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 969/1938] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:44.337864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed12800-6f25-4fcd-a6d3-6340af9dae1b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.348185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 970/1939] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:44 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:44.352610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed12800-6f25-4fcd-a6d3-6340af9dae1b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.352610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be861b7-ddee-4625-b559-a0f2744b47bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.353153 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be861b7-ddee-4625-b559-a0f2744b47bb admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.353435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be861b7-ddee-4625-b559-a0f2744b47bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.353708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be861b7-ddee-4625-b559-a0f2744b47bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.354808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be861b7-ddee-4625-b559-a0f2744b47bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:44.355117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be861b7-ddee-4625-b559-a0f2744b47bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.355446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254dea5-63b7-47b0-ade0-c4656938c803 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.356772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be861b7-ddee-4625-b559-a0f2744b47bb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.357060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7254dea5-63b7-47b0-ade0-c4656938c803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:44.366471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:44.372938 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 970/1940] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:42:44.373080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254dea5-63b7-47b0-ade0-c4656938c803 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.374409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be861b7-ddee-4625-b559-a0f2744b47bb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.375237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9c02-819b-45bb-b59c-1d78c68e0fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.375859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9c02-819b-45bb-b59c-1d78c68e0fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.376173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9c02-819b-45bb-b59c-1d78c68e0fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.381610 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9c02-819b-45bb-b59c-1d78c68e0fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.381610 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d9c02-819b-45bb-b59c-1d78c68e0fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.381610 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d9c02-819b-45bb-b59c-1d78c68e0fb9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.385434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be861b7-ddee-4625-b559-a0f2744b47bb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.392026 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 971/1941] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:44.392172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b03af-5edc-4180-b512-ca114c1d73b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.392172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b03af-5edc-4180-b512-ca114c1d73b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.392172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b03af-5edc-4180-b512-ca114c1d73b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.392172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b03af-5edc-4180-b512-ca114c1d73b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.392539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b03af-5edc-4180-b512-ca114c1d73b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.394429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b03af-5edc-4180-b512-ca114c1d73b9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.396884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9c02-819b-45bb-b59c-1d78c68e0fb9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.398887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:44.407086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9c02-819b-45bb-b59c-1d78c68e0fb9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.411834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26fd24-d2c7-4b0c-8a33-48c3c2eafbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.412827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26fd24-d2c7-4b0c-8a33-48c3c2eafbd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.413160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26fd24-d2c7-4b0c-8a33-48c3c2eafbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.413434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26fd24-d2c7-4b0c-8a33-48c3c2eafbd5 admin admin] PATH_INFO: `/v3/projects/b5caaddac38d4e7281ee03c3bebed66a/users/4e142a38c83c49899c4df58f6c010846/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.429404 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26fd24-d2c7-4b0c-8a33-48c3c2eafbd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5caaddac38d4e7281ee03c3bebed66a, user_id=4e142a38c83c49899c4df58f6c010846, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.432140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26fd24-d2c7-4b0c-8a33-48c3c2eafbd5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.513147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 971/1942] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:44 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:44.519085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b03af-5edc-4180-b512-ca114c1d73b9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.542325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 972/1943] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:44 2022] PUT /identity/v3/projects/b5caaddac38d4e7281ee03c3bebed66a/users/4e142a38c83c49899c4df58f6c010846/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:44.546916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26fd24-d2c7-4b0c-8a33-48c3c2eafbd5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.589040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b03af-5edc-4180-b512-ca114c1d73b9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.609842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26fd24-d2c7-4b0c-8a33-48c3c2eafbd5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.614751 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df41884-f6ac-4b39-b54c-26328b23c5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.615474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df41884-f6ac-4b39-b54c-26328b23c5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.615765 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df41884-f6ac-4b39-b54c-26328b23c5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.616106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df41884-f6ac-4b39-b54c-26328b23c5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.616809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df41884-f6ac-4b39-b54c-26328b23c5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.618453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df41884-f6ac-4b39-b54c-26328b23c5e4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.629366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 973/1944] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:44.633616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df41884-f6ac-4b39-b54c-26328b23c5e4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.644479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df41884-f6ac-4b39-b54c-26328b23c5e4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.651865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f04026-179d-4bb9-a4dd-d9e219fea2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.651865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04026-179d-4bb9-a4dd-d9e219fea2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.651865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04026-179d-4bb9-a4dd-d9e219fea2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.652387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04026-179d-4bb9-a4dd-d9e219fea2a7 admin admin] PATH_INFO: `/v3/projects/b5caaddac38d4e7281ee03c3bebed66a/users/4e142a38c83c49899c4df58f6c010846/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.653450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6776c58-8672-48df-8b07-a94d9e4890ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.654067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776c58-8672-48df-8b07-a94d9e4890ed admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.654351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776c58-8672-48df-8b07-a94d9e4890ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.654624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776c58-8672-48df-8b07-a94d9e4890ed admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.656353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6776c58-8672-48df-8b07-a94d9e4890ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.657325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f04026-179d-4bb9-a4dd-d9e219fea2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5caaddac38d4e7281ee03c3bebed66a, user_id=4e142a38c83c49899c4df58f6c010846, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.657810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6776c58-8672-48df-8b07-a94d9e4890ed admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.660341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f04026-179d-4bb9-a4dd-d9e219fea2a7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.672033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 974/1945] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:44 2022] PUT /identity/v3/projects/b5caaddac38d4e7281ee03c3bebed66a/users/4e142a38c83c49899c4df58f6c010846/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:44.683875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fed72f-cc51-4d0e-bd46-e737b5ad561a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.684278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fed72f-cc51-4d0e-bd46-e737b5ad561a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.684559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fed72f-cc51-4d0e-bd46-e737b5ad561a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.725874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f6fed72f-cc51-4d0e-bd46-e737b5ad561a None None] MFA Rules not processed for user `4e142a38c83c49899c4df58f6c010846`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:44.727730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f6fed72f-cc51-4d0e-bd46-e737b5ad561a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:44.731747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 972/1946] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:44 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:44.736538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6776c58-8672-48df-8b07-a94d9e4890ed admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.746470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6776c58-8672-48df-8b07-a94d9e4890ed admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.801552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 975/1947] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:44.810136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc1f14-230d-48ae-a5c2-2a96ac17c145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.810772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc1f14-230d-48ae-a5c2-2a96ac17c145 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.811163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc1f14-230d-48ae-a5c2-2a96ac17c145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.811635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc1f14-230d-48ae-a5c2-2a96ac17c145 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.812489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc1f14-230d-48ae-a5c2-2a96ac17c145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.822116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc1f14-230d-48ae-a5c2-2a96ac17c145 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.822116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:44.822661 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fed72f-cc51-4d0e-bd46-e737b5ad561a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.823628 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f6fed72f-cc51-4d0e-bd46-e737b5ad561a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:44.845857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 973/1948] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:44.847854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc1f14-230d-48ae-a5c2-2a96ac17c145 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.863837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc1f14-230d-48ae-a5c2-2a96ac17c145 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.872104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f7d58-0a3a-4a01-8da1-450dc06e8fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.873383 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f7d58-0a3a-4a01-8da1-450dc06e8fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.873383 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f7d58-0a3a-4a01-8da1-450dc06e8fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.873383 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f7d58-0a3a-4a01-8da1-450dc06e8fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.874221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f7d58-0a3a-4a01-8da1-450dc06e8fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.875834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f7d58-0a3a-4a01-8da1-450dc06e8fe4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.875948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:44.891419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 974/1949] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:44.894169 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fed72f-cc51-4d0e-bd46-e737b5ad561a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.894958 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f7d58-0a3a-4a01-8da1-450dc06e8fe4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:44.898952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da68885-41e8-4f4f-ad33-8a0d9e7faf8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.899833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da68885-41e8-4f4f-ad33-8a0d9e7faf8c service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.900336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da68885-41e8-4f4f-ad33-8a0d9e7faf8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.900823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da68885-41e8-4f4f-ad33-8a0d9e7faf8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.907857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f7d58-0a3a-4a01-8da1-450dc06e8fe4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:44.915318 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01922a1b-8b7a-4fdd-9d20-0f1dbae42c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:44.916422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da68885-41e8-4f4f-ad33-8a0d9e7faf8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.918627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01922a1b-8b7a-4fdd-9d20-0f1dbae42c62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:44.918824 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01922a1b-8b7a-4fdd-9d20-0f1dbae42c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:44.919105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01922a1b-8b7a-4fdd-9d20-0f1dbae42c62 admin admin] PATH_INFO: `/v3/projects/bb983c8f50d74133be00c897b85d12b2/users/f895babdb4f84350a8d1ea79c13f2074/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:44.920075 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da68885-41e8-4f4f-ad33-8a0d9e7faf8c service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.945343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01922a1b-8b7a-4fdd-9d20-0f1dbae42c62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb983c8f50d74133be00c897b85d12b2, user_id=f895babdb4f84350a8d1ea79c13f2074, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:44.948396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01922a1b-8b7a-4fdd-9d20-0f1dbae42c62 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:44.954523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 976/1950] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:42:45.036304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 975/1951] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:44 2022] PUT /identity/v3/projects/bb983c8f50d74133be00c897b85d12b2/users/f895babdb4f84350a8d1ea79c13f2074/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:45.046955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0086ac-e754-48e5-93e8-7d94b10640e1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:45.047528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0086ac-e754-48e5-93e8-7d94b10640e1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:45.047825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0086ac-e754-48e5-93e8-7d94b10640e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:45.125394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8c0086ac-e754-48e5-93e8-7d94b10640e1 None None] MFA Rules not processed for user `f895babdb4f84350a8d1ea79c13f2074`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:45.130442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8c0086ac-e754-48e5-93e8-7d94b10640e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:45.215713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01922a1b-8b7a-4fdd-9d20-0f1dbae42c62 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:45.219501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-01922a1b-8b7a-4fdd-9d20-0f1dbae42c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:45.243364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 977/1952] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:45.249943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0086ac-e754-48e5-93e8-7d94b10640e1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:45.293479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01922a1b-8b7a-4fdd-9d20-0f1dbae42c62 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:45.298611 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f59b0a-a6d6-4cb6-bcea-1b62c0842d93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:45.299265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f59b0a-a6d6-4cb6-bcea-1b62c0842d93 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:45.299547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f59b0a-a6d6-4cb6-bcea-1b62c0842d93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:45.300347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f59b0a-a6d6-4cb6-bcea-1b62c0842d93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:45.302183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b5f59b0a-a6d6-4cb6-bcea-1b62c0842d93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:45.339079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0086ac-e754-48e5-93e8-7d94b10640e1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:45.341918 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e24b4-9673-44c6-8acf-253648014768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:45.342577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e24b4-9673-44c6-8acf-253648014768 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:45.342862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e24b4-9673-44c6-8acf-253648014768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:45.343137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e24b4-9673-44c6-8acf-253648014768 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:45.343813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e24b4-9673-44c6-8acf-253648014768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:45.345801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e24b4-9673-44c6-8acf-253648014768 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:45.372769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f59b0a-a6d6-4cb6-bcea-1b62c0842d93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:45.374889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f59b0a-a6d6-4cb6-bcea-1b62c0842d93 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:45.383851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 976/1953] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:42:45.444700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 978/1954] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:45 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:45.445812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f59b0a-a6d6-4cb6-bcea-1b62c0842d93 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:45.451318 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:45.457570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f59b0a-a6d6-4cb6-bcea-1b62c0842d93 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:45.516754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb32ca4-3919-4751-af0b-67d03b34641c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:45.517425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb32ca4-3919-4751-af0b-67d03b34641c admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:45.517729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb32ca4-3919-4751-af0b-67d03b34641c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:45.517993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb32ca4-3919-4751-af0b-67d03b34641c admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:45.518679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb32ca4-3919-4751-af0b-67d03b34641c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:45.520679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb32ca4-3919-4751-af0b-67d03b34641c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:45.622844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 977/1955] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:45 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:45.627724 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e24b4-9673-44c6-8acf-253648014768 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:45.641221 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e24b4-9673-44c6-8acf-253648014768 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:45.647022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4679b-b6c7-483b-bd15-3cb166cd51d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:45.647846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4679b-b6c7-483b-bd15-3cb166cd51d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:45.648168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4679b-b6c7-483b-bd15-3cb166cd51d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:45.648508 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4679b-b6c7-483b-bd15-3cb166cd51d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:45.649289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4679b-b6c7-483b-bd15-3cb166cd51d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:45.650971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4679b-b6c7-483b-bd15-3cb166cd51d4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:45.662759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 979/1956] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:45.668152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb32ca4-3919-4751-af0b-67d03b34641c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:45.679545 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb32ca4-3919-4751-af0b-67d03b34641c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:45.684197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e82b2-6fff-414b-a676-222c60d07efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:45.684862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e82b2-6fff-414b-a676-222c60d07efc admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:45.685329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e82b2-6fff-414b-a676-222c60d07efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:45.685614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e82b2-6fff-414b-a676-222c60d07efc admin admin] PATH_INFO: `/v3/projects/f8d4ccce2f6f480ebd2726f92997e2b3/users/0dafc79f1b714a9796a464bbf5d41533/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:45.703099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e82b2-6fff-414b-a676-222c60d07efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d4ccce2f6f480ebd2726f92997e2b3, user_id=0dafc79f1b714a9796a464bbf5d41533, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:45.705812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e82b2-6fff-414b-a676-222c60d07efc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:45.792062 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 978/1957] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:45 2022] PUT /identity/v3/projects/f8d4ccce2f6f480ebd2726f92997e2b3/users/0dafc79f1b714a9796a464bbf5d41533/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:45.796749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4679b-b6c7-483b-bd15-3cb166cd51d4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:45.806819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4679b-b6c7-483b-bd15-3cb166cd51d4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:45.853916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:45.881202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263d50e-2e96-4cc1-874d-a15ebb3ec88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:45.882014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263d50e-2e96-4cc1-874d-a15ebb3ec88e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:45.882296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263d50e-2e96-4cc1-874d-a15ebb3ec88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:45.882581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263d50e-2e96-4cc1-874d-a15ebb3ec88e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:45.883280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263d50e-2e96-4cc1-874d-a15ebb3ec88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:45.884930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263d50e-2e96-4cc1-874d-a15ebb3ec88e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:45.896789 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 980/1958] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:45.901178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e82b2-6fff-414b-a676-222c60d07efc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:45.912487 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e82b2-6fff-414b-a676-222c60d07efc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:45.917056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59813f8e-a00f-4ce7-87e8-0df37ab9b076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:45.917771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59813f8e-a00f-4ce7-87e8-0df37ab9b076 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:45.918053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59813f8e-a00f-4ce7-87e8-0df37ab9b076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:45.918329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59813f8e-a00f-4ce7-87e8-0df37ab9b076 admin admin] PATH_INFO: `/v3/projects/f8d4ccce2f6f480ebd2726f92997e2b3/users/0dafc79f1b714a9796a464bbf5d41533/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:45.920302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59813f8e-a00f-4ce7-87e8-0df37ab9b076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d4ccce2f6f480ebd2726f92997e2b3, user_id=0dafc79f1b714a9796a464bbf5d41533, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:45.923934 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59813f8e-a00f-4ce7-87e8-0df37ab9b076 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:45.937989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 979/1959] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:45 2022] PUT /identity/v3/projects/f8d4ccce2f6f480ebd2726f92997e2b3/users/0dafc79f1b714a9796a464bbf5d41533/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:45.949300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cdbd7-210c-465e-95b4-63c691db3e9a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:45.949588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cdbd7-210c-465e-95b4-63c691db3e9a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:45.949865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cdbd7-210c-465e-95b4-63c691db3e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:45.979274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f07cdbd7-210c-465e-95b4-63c691db3e9a None None] MFA Rules not processed for user `0dafc79f1b714a9796a464bbf5d41533`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:45.980551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f07cdbd7-210c-465e-95b4-63c691db3e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:46.088316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 981/1960] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:46.102509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59813f8e-a00f-4ce7-87e8-0df37ab9b076 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:46.215534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59813f8e-a00f-4ce7-87e8-0df37ab9b076 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:46.220534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544834fc-85fe-46f1-aab3-3a6da9abb335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:46.221173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544834fc-85fe-46f1-aab3-3a6da9abb335 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:46.221689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544834fc-85fe-46f1-aab3-3a6da9abb335 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:46.221974 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544834fc-85fe-46f1-aab3-3a6da9abb335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:46.233039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544834fc-85fe-46f1-aab3-3a6da9abb335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:46.235085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544834fc-85fe-46f1-aab3-3a6da9abb335 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:46.246370 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 980/1961] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:46.403172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cdbd7-210c-465e-95b4-63c691db3e9a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:46.413230 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cdbd7-210c-465e-95b4-63c691db3e9a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:46.416926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bc205-a521-4390-ab27-f4da9fd69672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:46.417547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bc205-a521-4390-ab27-f4da9fd69672 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:46.417824 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bc205-a521-4390-ab27-f4da9fd69672 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:46.418093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bc205-a521-4390-ab27-f4da9fd69672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:46.483769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80bc205-a521-4390-ab27-f4da9fd69672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:46.485462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80bc205-a521-4390-ab27-f4da9fd69672 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:46.497845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 982/1962] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:51.980449 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544834fc-85fe-46f1-aab3-3a6da9abb335 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:52.045924 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:52.050383 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544834fc-85fe-46f1-aab3-3a6da9abb335 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:52.055154 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:52.055807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:52.056129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:52.056430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] PATH_INFO: `/v3/users/f895babdb4f84350a8d1ea79c13f2074` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:52.057594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f895babdb4f84350a8d1ea79c13f2074)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:52.059510 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:52.274346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f895babdb4f84350a8d1ea79c13f2074'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:52.275504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f895babdb4f84350a8d1ea79c13f2074'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:52.276036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f895babdb4f84350a8d1ea79c13f2074'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:52.294758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f895babdb4f84350a8d1ea79c13f2074'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:52.375249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f895babdb4f84350a8d1ea79c13f2074'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:52.388835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 981/1963] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:51 2022] DELETE /identity/v3/users/f895babdb4f84350a8d1ea79c13f2074 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:52.393397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4461d5c-e010-4f27-b7fa-c57848dd2b84 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:52.393861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4461d5c-e010-4f27-b7fa-c57848dd2b84 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:52.393950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4461d5c-e010-4f27-b7fa-c57848dd2b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:52.414626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b4461d5c-e010-4f27-b7fa-c57848dd2b84 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:52.414836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b4461d5c-e010-4f27-b7fa-c57848dd2b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:52.494237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 983/1964] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:52.511212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:52.512658 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:52.572524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0f9aa-5163-4593-b58e-34e8e91f88d5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:52.576277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75715e8d-d86f-4533-bbfb-17c127f6f6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:52.576934 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75715e8d-d86f-4533-bbfb-17c127f6f6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:52.577225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75715e8d-d86f-4533-bbfb-17c127f6f6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:52.577505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75715e8d-d86f-4533-bbfb-17c127f6f6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:52.586821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75715e8d-d86f-4533-bbfb-17c127f6f6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:52.588296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75715e8d-d86f-4533-bbfb-17c127f6f6b3 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:52.600339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 982/1965] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:53.103180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4461d5c-e010-4f27-b7fa-c57848dd2b84 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:53.105984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b4461d5c-e010-4f27-b7fa-c57848dd2b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:53.133905 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:53.138518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4461d5c-e010-4f27-b7fa-c57848dd2b84 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:53.143947 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:53.144789 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:53.149627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:53.149627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] PATH_INFO: `/v3/projects/bb983c8f50d74133be00c897b85d12b2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:53.149627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb983c8f50d74133be00c897b85d12b2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:53.149627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:53.278539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb983c8f50d74133be00c897b85d12b2', 'request_id': 'req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:53.368990 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb983c8f50d74133be00c897b85d12b2', 'request_id': 'req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:53.376420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb983c8f50d74133be00c897b85d12b2', 'request_id': 'req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:53.388531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] The token cache is being invalidate because project bb983c8f50d74133be00c897b85d12b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:53.388892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:53.389684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:53.391270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 984/1966] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:53 2022] DELETE /identity/v3/projects/bb983c8f50d74133be00c897b85d12b2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:53.395363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75715e8d-d86f-4533-bbfb-17c127f6f6b3 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:53.405549 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75715e8d-d86f-4533-bbfb-17c127f6f6b3 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:53.472260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:53.473558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:53.473725 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:53.474013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] PATH_INFO: `/v3/users/0dafc79f1b714a9796a464bbf5d41533` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:53.475961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dafc79f1b714a9796a464bbf5d41533)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:53.478623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:53.510053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:53.699523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dafc79f1b714a9796a464bbf5d41533'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:53.700524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dafc79f1b714a9796a464bbf5d41533'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:53.701963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dafc79f1b714a9796a464bbf5d41533'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:53.725447 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dafc79f1b714a9796a464bbf5d41533'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:53.849335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dafc79f1b714a9796a464bbf5d41533'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:53.865529 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 983/1967] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:53 2022] DELETE /identity/v3/users/0dafc79f1b714a9796a464bbf5d41533 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:54.458853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:54.460346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:54.482539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a28d93-3ce6-4cef-b4ec-dfc17c2bed7e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:54.507562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:54.509327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:54.548941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:54.549662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:54.549950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:54.550237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] PATH_INFO: `/v3/projects/f8d4ccce2f6f480ebd2726f92997e2b3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:54.551507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d4ccce2f6f480ebd2726f92997e2b3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:54.553649 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:54.580851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef8a9f-59cf-4c55-bcd2-867ee83d3ee6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:54.586184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2865330-f2ab-4a4b-926e-0e87f881d512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:54.586899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2865330-f2ab-4a4b-926e-0e87f881d512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:54.587112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2865330-f2ab-4a4b-926e-0e87f881d512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:54.587389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2865330-f2ab-4a4b-926e-0e87f881d512 admin admin] PATH_INFO: `/v3/users/dac84df2824b444eb2cfbdd6a576f24d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:54.588570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2865330-f2ab-4a4b-926e-0e87f881d512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac84df2824b444eb2cfbdd6a576f24d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:54.590483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2865330-f2ab-4a4b-926e-0e87f881d512 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:54.690871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d4ccce2f6f480ebd2726f92997e2b3', 'request_id': 'req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:54.816237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d4ccce2f6f480ebd2726f92997e2b3', 'request_id': 'req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:54.822498 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d4ccce2f6f480ebd2726f92997e2b3', 'request_id': 'req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:54.834176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] The token cache is being invalidate because project f8d4ccce2f6f480ebd2726f92997e2b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:54.834469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:54.835105 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4d381ef-58a8-4b64-b5f2-24706aa82f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:54.836661 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 985/1968] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:54 2022] DELETE /identity/v3/projects/f8d4ccce2f6f480ebd2726f92997e2b3 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:54.841693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc38dc-07fd-4345-a8e4-e961a5b7501c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:54.842411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc38dc-07fd-4345-a8e4-e961a5b7501c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:54.842688 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc38dc-07fd-4345-a8e4-e961a5b7501c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:54.865089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a5bc38dc-07fd-4345-a8e4-e961a5b7501c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:54.866444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5bc38dc-07fd-4345-a8e4-e961a5b7501c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:54.881094 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:54.889255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a2865330-f2ab-4a4b-926e-0e87f881d512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac84df2824b444eb2cfbdd6a576f24d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:54.890060 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a2865330-f2ab-4a4b-926e-0e87f881d512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac84df2824b444eb2cfbdd6a576f24d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:54.890988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a2865330-f2ab-4a4b-926e-0e87f881d512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac84df2824b444eb2cfbdd6a576f24d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:54.903876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a2865330-f2ab-4a4b-926e-0e87f881d512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac84df2824b444eb2cfbdd6a576f24d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:54.932792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 986/1969] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:54.940996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03630533-033b-4174-bcd6-4ec8dee735eb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:54.941274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03630533-033b-4174-bcd6-4ec8dee735eb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:54.941549 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03630533-033b-4174-bcd6-4ec8dee735eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:54.967635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-03630533-033b-4174-bcd6-4ec8dee735eb None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:54.967635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-03630533-033b-4174-bcd6-4ec8dee735eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:54.974409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 987/1970] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:54.983493 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03630533-033b-4174-bcd6-4ec8dee735eb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:54.987667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a2865330-f2ab-4a4b-926e-0e87f881d512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac84df2824b444eb2cfbdd6a576f24d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:54.993951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03630533-033b-4174-bcd6-4ec8dee735eb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:54.999731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac069578-1f9e-46d6-8ce1-e094a211c170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:55.000400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac069578-1f9e-46d6-8ce1-e094a211c170 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.000686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac069578-1f9e-46d6-8ce1-e094a211c170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.000966 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac069578-1f9e-46d6-8ce1-e094a211c170 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.001670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac069578-1f9e-46d6-8ce1-e094a211c170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:55.001978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac069578-1f9e-46d6-8ce1-e094a211c170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:55.003584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac069578-1f9e-46d6-8ce1-e094a211c170 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:55.004570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 984/1971] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:54 2022] DELETE /identity/v3/users/dac84df2824b444eb2cfbdd6a576f24d => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:55.008909 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbeabfe-c4a6-4b78-8757-2636c9071532 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.009526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbeabfe-c4a6-4b78-8757-2636c9071532 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.009526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbeabfe-c4a6-4b78-8757-2636c9071532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.020060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 988/1972] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:42:55.021111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac069578-1f9e-46d6-8ce1-e094a211c170 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:55.030560 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2bbeabfe-c4a6-4b78-8757-2636c9071532 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:55.031875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2bbeabfe-c4a6-4b78-8757-2636c9071532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:55.036192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac069578-1f9e-46d6-8ce1-e094a211c170 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:55.051522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:55.130253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74116671-dc9b-4412-bcb9-51d0017eed26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:55.131049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74116671-dc9b-4412-bcb9-51d0017eed26 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.131303 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 985/1973] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:55.131410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74116671-dc9b-4412-bcb9-51d0017eed26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.131692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74116671-dc9b-4412-bcb9-51d0017eed26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.132510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74116671-dc9b-4412-bcb9-51d0017eed26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:55.135545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74116671-dc9b-4412-bcb9-51d0017eed26 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:55.142025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbeabfe-c4a6-4b78-8757-2636c9071532 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:55.143524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2bbeabfe-c4a6-4b78-8757-2636c9071532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:55.209499 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbeabfe-c4a6-4b78-8757-2636c9071532 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:55.213782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a627484-e9cd-45b7-823d-f6eaca26be80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:55.214421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a627484-e9cd-45b7-823d-f6eaca26be80 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.214734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a627484-e9cd-45b7-823d-f6eaca26be80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.215011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a627484-e9cd-45b7-823d-f6eaca26be80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.219320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 989/1974] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:55 2022] POST /identity/v3/projects => generated 389 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:55.223239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74116671-dc9b-4412-bcb9-51d0017eed26 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:55.227671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a627484-e9cd-45b7-823d-f6eaca26be80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:55.229343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a627484-e9cd-45b7-823d-f6eaca26be80 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:55.241171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 986/1975] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:55.298905 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74116671-dc9b-4412-bcb9-51d0017eed26 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:55.304377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de068a8a-1584-4f64-89d0-36a0363958f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:55.305041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de068a8a-1584-4f64-89d0-36a0363958f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.305391 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de068a8a-1584-4f64-89d0-36a0363958f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.305738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de068a8a-1584-4f64-89d0-36a0363958f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.306528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de068a8a-1584-4f64-89d0-36a0363958f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:55.308793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de068a8a-1584-4f64-89d0-36a0363958f0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:55.319459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a627484-e9cd-45b7-823d-f6eaca26be80 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:55.320986 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0a627484-e9cd-45b7-823d-f6eaca26be80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:55.336885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a627484-e9cd-45b7-823d-f6eaca26be80 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:55.343415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded610b-2097-4276-950a-5bdb437ff1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:55.346593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded610b-2097-4276-950a-5bdb437ff1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.346593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded610b-2097-4276-950a-5bdb437ff1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.346593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded610b-2097-4276-950a-5bdb437ff1cf admin admin] PATH_INFO: `/v3/users/0b9e8486eec34deba9bc024067fc2524` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.346593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded610b-2097-4276-950a-5bdb437ff1cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b9e8486eec34deba9bc024067fc2524)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:55.348016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded610b-2097-4276-950a-5bdb437ff1cf admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:55.412049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 990/1976] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:55 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:55.416094 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de068a8a-1584-4f64-89d0-36a0363958f0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:55.425781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de068a8a-1584-4f64-89d0-36a0363958f0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:55.430470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604bf31-45be-49ca-95ec-6267003a074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:55.431142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604bf31-45be-49ca-95ec-6267003a074e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.431492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604bf31-45be-49ca-95ec-6267003a074e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.431836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604bf31-45be-49ca-95ec-6267003a074e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.432732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604bf31-45be-49ca-95ec-6267003a074e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:55.434293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604bf31-45be-49ca-95ec-6267003a074e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:55.447705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 991/1977] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:55.448948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604bf31-45be-49ca-95ec-6267003a074e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:55.455583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:55.461209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604bf31-45be-49ca-95ec-6267003a074e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:55.467790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a324f0-ee61-4d17-8336-18dd1a31be08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:55.468020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a324f0-ee61-4d17-8336-18dd1a31be08 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.468288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a324f0-ee61-4d17-8336-18dd1a31be08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.468569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a324f0-ee61-4d17-8336-18dd1a31be08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.469286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a324f0-ee61-4d17-8336-18dd1a31be08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:55.470945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a324f0-ee61-4d17-8336-18dd1a31be08 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:55.486270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 992/1978] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:55.490484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a324f0-ee61-4d17-8336-18dd1a31be08 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:55.501205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a324f0-ee61-4d17-8336-18dd1a31be08 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:55.505961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671b979-0707-455f-a32e-7f1f05821e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:55.506633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671b979-0707-455f-a32e-7f1f05821e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.506918 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671b979-0707-455f-a32e-7f1f05821e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.508018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671b979-0707-455f-a32e-7f1f05821e2c admin admin] PATH_INFO: `/v3/projects/07715d4acc3c44548473850a510a8660/users/b38d9af2070440029006697fb0900294/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.522898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671b979-0707-455f-a32e-7f1f05821e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=07715d4acc3c44548473850a510a8660, user_id=b38d9af2070440029006697fb0900294, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:55.525620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671b979-0707-455f-a32e-7f1f05821e2c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:55.555605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dded610b-2097-4276-950a-5bdb437ff1cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b9e8486eec34deba9bc024067fc2524'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:55.556270 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dded610b-2097-4276-950a-5bdb437ff1cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b9e8486eec34deba9bc024067fc2524'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:55.556904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dded610b-2097-4276-950a-5bdb437ff1cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b9e8486eec34deba9bc024067fc2524'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:55.571235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dded610b-2097-4276-950a-5bdb437ff1cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b9e8486eec34deba9bc024067fc2524'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:55.603379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 993/1979] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:55 2022] PUT /identity/v3/projects/07715d4acc3c44548473850a510a8660/users/b38d9af2070440029006697fb0900294/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:55.615240 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f7900-c850-4171-9ba4-a29fb1773f6b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.615471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f7900-c850-4171-9ba4-a29fb1773f6b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.615756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f7900-c850-4171-9ba4-a29fb1773f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.645561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-010f7900-c850-4171-9ba4-a29fb1773f6b None None] MFA Rules not processed for user `b38d9af2070440029006697fb0900294`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:55.646759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-010f7900-c850-4171-9ba4-a29fb1773f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:55.662190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dded610b-2097-4276-950a-5bdb437ff1cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b9e8486eec34deba9bc024067fc2524'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:55.680597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 987/1980] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:55 2022] DELETE /identity/v3/users/0b9e8486eec34deba9bc024067fc2524 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:55.686053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87819-c39a-426f-94f2-57876094ad9e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.686361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87819-c39a-426f-94f2-57876094ad9e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.686659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87819-c39a-426f-94f2-57876094ad9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.707782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8db87819-c39a-426f-94f2-57876094ad9e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:55.709007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8db87819-c39a-426f-94f2-57876094ad9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:55.721338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 994/1981] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:55.725636 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f7900-c850-4171-9ba4-a29fb1773f6b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:55.727546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-010f7900-c850-4171-9ba4-a29fb1773f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:55.745174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:55.783271 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 988/1982] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:55.794777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db87819-c39a-426f-94f2-57876094ad9e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:55.796213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8db87819-c39a-426f-94f2-57876094ad9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:55.804256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f7900-c850-4171-9ba4-a29fb1773f6b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:55.808985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8adb48-a179-4d7a-b1b0-e6a03740ef7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:55.809621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8adb48-a179-4d7a-b1b0-e6a03740ef7d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.809890 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8adb48-a179-4d7a-b1b0-e6a03740ef7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.810172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8adb48-a179-4d7a-b1b0-e6a03740ef7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.811558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8adb48-a179-4d7a-b1b0-e6a03740ef7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:55.813516 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8adb48-a179-4d7a-b1b0-e6a03740ef7d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:55.858618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db87819-c39a-426f-94f2-57876094ad9e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:55.862460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba234f23-89e0-4f1b-98db-90a0b530137e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:55.863110 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba234f23-89e0-4f1b-98db-90a0b530137e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:55.864018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba234f23-89e0-4f1b-98db-90a0b530137e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:55.864018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba234f23-89e0-4f1b-98db-90a0b530137e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:55.874624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba234f23-89e0-4f1b-98db-90a0b530137e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:55.876375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba234f23-89e0-4f1b-98db-90a0b530137e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:55.887334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 989/1983] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:42:55.914615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 995/1984] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:55 2022] POST /identity/v3/projects => generated 389 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:42:55.917474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba234f23-89e0-4f1b-98db-90a0b530137e service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:55.933668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba234f23-89e0-4f1b-98db-90a0b530137e service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:55.960343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8adb48-a179-4d7a-b1b0-e6a03740ef7d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:55.962407 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eb8adb48-a179-4d7a-b1b0-e6a03740ef7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:56.045473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20876a4-63a7-490c-8cc9-66b43cbfabfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:56.046508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20876a4-63a7-490c-8cc9-66b43cbfabfd admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:56.046872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20876a4-63a7-490c-8cc9-66b43cbfabfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:56.047485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20876a4-63a7-490c-8cc9-66b43cbfabfd admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:56.048918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20876a4-63a7-490c-8cc9-66b43cbfabfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:56.051585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20876a4-63a7-490c-8cc9-66b43cbfabfd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:56.069504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:56.090753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8adb48-a179-4d7a-b1b0-e6a03740ef7d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:56.095845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:56.096614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:56.096970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:56.097316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] PATH_INFO: `/v3/projects/be5b129c51a349b8aa872f04e5313e87` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:56.098686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=be5b129c51a349b8aa872f04e5313e87)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:56.100720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:56.192425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 990/1985] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:55 2022] POST /identity/v3/users => generated 301 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:56.193476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20876a4-63a7-490c-8cc9-66b43cbfabfd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:56.205258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20876a4-63a7-490c-8cc9-66b43cbfabfd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:56.210988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018e57d-d4c2-40e8-a2ab-0ef97dc7bd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:56.211841 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018e57d-d4c2-40e8-a2ab-0ef97dc7bd88 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:56.212277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018e57d-d4c2-40e8-a2ab-0ef97dc7bd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:56.215156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018e57d-d4c2-40e8-a2ab-0ef97dc7bd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:56.215156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018e57d-d4c2-40e8-a2ab-0ef97dc7bd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:56.215156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018e57d-d4c2-40e8-a2ab-0ef97dc7bd88 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:56.228506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be5b129c51a349b8aa872f04e5313e87', 'request_id': 'req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.228815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 991/1986] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:56.233571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018e57d-d4c2-40e8-a2ab-0ef97dc7bd88 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:56.244318 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018e57d-d4c2-40e8-a2ab-0ef97dc7bd88 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:56.309007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be5b129c51a349b8aa872f04e5313e87', 'request_id': 'req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.312683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919c906-18ea-4aa9-9355-1ae9dab0c2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:56.313457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919c906-18ea-4aa9-9355-1ae9dab0c2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:56.320429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919c906-18ea-4aa9-9355-1ae9dab0c2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:56.320429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919c906-18ea-4aa9-9355-1ae9dab0c2b9 admin admin] PATH_INFO: `/v3/projects/397f7a4ce45048618e6c46d1becd8744/users/2c0602637b5449e9905d429bc26f02e9/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:56.323129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be5b129c51a349b8aa872f04e5313e87', 'request_id': 'req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.332029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919c906-18ea-4aa9-9355-1ae9dab0c2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=397f7a4ce45048618e6c46d1becd8744, user_id=2c0602637b5449e9905d429bc26f02e9, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:56.334880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919c906-18ea-4aa9-9355-1ae9dab0c2b9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:56.339543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] The token cache is being invalidate because project be5b129c51a349b8aa872f04e5313e87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:56.339923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.341078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.342997 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 996/1987] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:55 2022] DELETE /identity/v3/projects/be5b129c51a349b8aa872f04e5313e87 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:56.347308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:56.348877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:56.362472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be1eef-5c89-42ba-b91b-5bb69dc3be1a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:56.368546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4162103-308d-493c-9421-51958002ca6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:56.369199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4162103-308d-493c-9421-51958002ca6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:56.369485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4162103-308d-493c-9421-51958002ca6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:56.369767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4162103-308d-493c-9421-51958002ca6b admin admin] PATH_INFO: `/v3/users/4e142a38c83c49899c4df58f6c010846` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:56.371152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4162103-308d-493c-9421-51958002ca6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e142a38c83c49899c4df58f6c010846)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:56.373158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4162103-308d-493c-9421-51958002ca6b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:56.411597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 992/1988] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:56 2022] PUT /identity/v3/projects/397f7a4ce45048618e6c46d1becd8744/users/2c0602637b5449e9905d429bc26f02e9/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:56.420891 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919c906-18ea-4aa9-9355-1ae9dab0c2b9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:56.420891 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d919c906-18ea-4aa9-9355-1ae9dab0c2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:56.484650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919c906-18ea-4aa9-9355-1ae9dab0c2b9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:56.489791 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93395c9-b169-4ae4-8766-55e3ca68fee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:56.490483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93395c9-b169-4ae4-8766-55e3ca68fee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:56.490773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93395c9-b169-4ae4-8766-55e3ca68fee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:56.491049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93395c9-b169-4ae4-8766-55e3ca68fee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:56.491800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93395c9-b169-4ae4-8766-55e3ca68fee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:56.493688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93395c9-b169-4ae4-8766-55e3ca68fee3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:56.505352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 993/1989] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:56.509604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93395c9-b169-4ae4-8766-55e3ca68fee3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:56.523299 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93395c9-b169-4ae4-8766-55e3ca68fee3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:56.535883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99624-9788-426b-8ce0-56d0dc4e8dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:56.536594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99624-9788-426b-8ce0-56d0dc4e8dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:56.536881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99624-9788-426b-8ce0-56d0dc4e8dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:56.537330 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99624-9788-426b-8ce0-56d0dc4e8dd3 admin admin] PATH_INFO: `/v3/projects/397f7a4ce45048618e6c46d1becd8744/users/2c0602637b5449e9905d429bc26f02e9/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:56.539594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99624-9788-426b-8ce0-56d0dc4e8dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=397f7a4ce45048618e6c46d1becd8744, user_id=2c0602637b5449e9905d429bc26f02e9, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:56.542414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99624-9788-426b-8ce0-56d0dc4e8dd3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:56.573231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 994/1990] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:56 2022] PUT /identity/v3/projects/397f7a4ce45048618e6c46d1becd8744/users/2c0602637b5449e9905d429bc26f02e9/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:56.573231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99624-9788-426b-8ce0-56d0dc4e8dd3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:56.573231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-18f99624-9788-426b-8ce0-56d0dc4e8dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:56.600831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4162103-308d-493c-9421-51958002ca6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e142a38c83c49899c4df58f6c010846'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.601528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4162103-308d-493c-9421-51958002ca6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e142a38c83c49899c4df58f6c010846'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.612661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:56.612909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4162103-308d-493c-9421-51958002ca6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e142a38c83c49899c4df58f6c010846'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.628226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4162103-308d-493c-9421-51958002ca6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e142a38c83c49899c4df58f6c010846'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.665049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99624-9788-426b-8ce0-56d0dc4e8dd3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:56.670382 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:56.671188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:56.671570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:56.672018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] PATH_INFO: `/v3/projects/40675c49c8ae4a1d844e2702630a8d25` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:56.673686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40675c49c8ae4a1d844e2702630a8d25)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:56.675703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:56.719185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4162103-308d-493c-9421-51958002ca6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e142a38c83c49899c4df58f6c010846'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.727013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 997/1991] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:56 2022] DELETE /identity/v3/users/4e142a38c83c49899c4df58f6c010846 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:56.728339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81fb0c-2d61-494f-84d0-0760fb21eb51 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:56.729339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81fb0c-2d61-494f-84d0-0760fb21eb51 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:56.729849 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81fb0c-2d61-494f-84d0-0760fb21eb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:56.769170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5c81fb0c-2d61-494f-84d0-0760fb21eb51 None None] MFA Rules not processed for user `2c0602637b5449e9905d429bc26f02e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:56.770863 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5c81fb0c-2d61-494f-84d0-0760fb21eb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:56.815054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:56.822841 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40675c49c8ae4a1d844e2702630a8d25', 'request_id': 'req-359a386c-d0fb-4ee7-88f2-3f89a365b582'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.833125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40675c49c8ae4a1d844e2702630a8d25', 'request_id': 'req-359a386c-d0fb-4ee7-88f2-3f89a365b582'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.858992 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 998/1992] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:56.870186 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81fb0c-2d61-494f-84d0-0760fb21eb51 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:56.871624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5c81fb0c-2d61-494f-84d0-0760fb21eb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:56.913532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40675c49c8ae4a1d844e2702630a8d25', 'request_id': 'req-359a386c-d0fb-4ee7-88f2-3f89a365b582'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.926783 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] The token cache is being invalidate because project 40675c49c8ae4a1d844e2702630a8d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:56.927173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.927891 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:56.929617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 995/1993] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:56 2022] DELETE /identity/v3/projects/40675c49c8ae4a1d844e2702630a8d25 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:56.933150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81fb0c-2d61-494f-84d0-0760fb21eb51 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:56.933761 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:56.935199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:57.020031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f7d2b-dbe1-497e-b179-96724e778fe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:57.021151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f7d2b-dbe1-497e-b179-96724e778fe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:57.021618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f7d2b-dbe1-497e-b179-96724e778fe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:57.022063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f7d2b-dbe1-497e-b179-96724e778fe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:57.025862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eb0f7d2b-dbe1-497e-b179-96724e778fe0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:57.034656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a386c-d0fb-4ee7-88f2-3f89a365b582 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:57.048622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:57.049322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:57.049584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:57.049872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] PATH_INFO: `/v3/users/d38ad46f144f462eb45b915e2087ca9d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:57.051547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d38ad46f144f462eb45b915e2087ca9d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:57.054787 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:57.108333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:57.147664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f7d2b-dbe1-497e-b179-96724e778fe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:57.149393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f7d2b-dbe1-497e-b179-96724e778fe0 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:57.164691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 999/1994] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:56 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:42:57.226088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd38ad46f144f462eb45b915e2087ca9d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.227478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd38ad46f144f462eb45b915e2087ca9d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.228168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd38ad46f144f462eb45b915e2087ca9d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.243640 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd38ad46f144f462eb45b915e2087ca9d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.306018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f7d2b-dbe1-497e-b179-96724e778fe0 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:57.307625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eb0f7d2b-dbe1-497e-b179-96724e778fe0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:57.330107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd38ad46f144f462eb45b915e2087ca9d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.342287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f7d2b-dbe1-497e-b179-96724e778fe0 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:57.347059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 996/1995] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:56 2022] DELETE /identity/v3/users/d38ad46f144f462eb45b915e2087ca9d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:57.348599 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:57.351728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:57.352459 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:57.352748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:57.353015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] PATH_INFO: `/v3/projects/b5caaddac38d4e7281ee03c3bebed66a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:57.354212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5caaddac38d4e7281ee03c3bebed66a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:57.356173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:57.358239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:57.462964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd50ec8-075c-4105-a6cd-c120cb15940b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:57.469163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeae77a-04bf-4bf6-954b-0e9928fe18ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:57.470437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeae77a-04bf-4bf6-954b-0e9928fe18ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:57.470935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeae77a-04bf-4bf6-954b-0e9928fe18ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:57.471553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeae77a-04bf-4bf6-954b-0e9928fe18ae admin admin] PATH_INFO: `/v3/users/b38d9af2070440029006697fb0900294` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:57.473619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeae77a-04bf-4bf6-954b-0e9928fe18ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b38d9af2070440029006697fb0900294)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:57.475493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeae77a-04bf-4bf6-954b-0e9928fe18ae admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:57.502339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5caaddac38d4e7281ee03c3bebed66a', 'request_id': 'req-245db4fd-0c28-4fdb-b389-e18c90c5e502'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.604355 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5caaddac38d4e7281ee03c3bebed66a', 'request_id': 'req-245db4fd-0c28-4fdb-b389-e18c90c5e502'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.613246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5caaddac38d4e7281ee03c3bebed66a', 'request_id': 'req-245db4fd-0c28-4fdb-b389-e18c90c5e502'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.635257 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] The token cache is being invalidate because project b5caaddac38d4e7281ee03c3bebed66a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:57.639336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.639336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-245db4fd-0c28-4fdb-b389-e18c90c5e502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.644665 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1000/1996] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:57 2022] DELETE /identity/v3/projects/b5caaddac38d4e7281ee03c3bebed66a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:42:57.646363 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60a785-0e7d-46cb-a475-48696813fda9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:57.646716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60a785-0e7d-46cb-a475-48696813fda9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:57.647110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60a785-0e7d-46cb-a475-48696813fda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:57.673053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0b60a785-0e7d-46cb-a475-48696813fda9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:57.674315 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0b60a785-0e7d-46cb-a475-48696813fda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:57.730233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:57.761470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-abeae77a-04bf-4bf6-954b-0e9928fe18ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b38d9af2070440029006697fb0900294'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.762767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-abeae77a-04bf-4bf6-954b-0e9928fe18ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b38d9af2070440029006697fb0900294'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.763444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-abeae77a-04bf-4bf6-954b-0e9928fe18ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b38d9af2070440029006697fb0900294'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.798753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1001/1997] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:57.798986 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-abeae77a-04bf-4bf6-954b-0e9928fe18ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b38d9af2070440029006697fb0900294'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.800586 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feb5f2-9019-4999-847f-983e2edd712a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:57.800885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feb5f2-9019-4999-847f-983e2edd712a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:57.801167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feb5f2-9019-4999-847f-983e2edd712a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:57.822264 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-68feb5f2-9019-4999-847f-983e2edd712a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:57.823565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-68feb5f2-9019-4999-847f-983e2edd712a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:57.835399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1002/1998] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:57.839273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feb5f2-9019-4999-847f-983e2edd712a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:57.852658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feb5f2-9019-4999-847f-983e2edd712a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:57.861499 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13374fe-b131-4c28-b526-52273f1083cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:57.862349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13374fe-b131-4c28-b526-52273f1083cb admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:57.862684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13374fe-b131-4c28-b526-52273f1083cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:57.863102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13374fe-b131-4c28-b526-52273f1083cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:57.864280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13374fe-b131-4c28-b526-52273f1083cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:57.864428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13374fe-b131-4c28-b526-52273f1083cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:57.866168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13374fe-b131-4c28-b526-52273f1083cb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:57.872271 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-abeae77a-04bf-4bf6-954b-0e9928fe18ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b38d9af2070440029006697fb0900294'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:57.883708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1003/1999] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:42:57.907011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13374fe-b131-4c28-b526-52273f1083cb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:57.913860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 997/2000] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:57 2022] DELETE /identity/v3/users/b38d9af2070440029006697fb0900294 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:57.918914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d875d9-a481-48e7-b7d4-5fa1290a7161 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:57.919290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d875d9-a481-48e7-b7d4-5fa1290a7161 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:57.919562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d875d9-a481-48e7-b7d4-5fa1290a7161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:57.924922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13374fe-b131-4c28-b526-52273f1083cb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:57.972207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-11d875d9-a481-48e7-b7d4-5fa1290a7161 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:57.973791 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-11d875d9-a481-48e7-b7d4-5fa1290a7161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:58.074756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c55ad4-30c5-4d11-b67f-d6be42c72a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:58.075631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c55ad4-30c5-4d11-b67f-d6be42c72a56 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.075918 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c55ad4-30c5-4d11-b67f-d6be42c72a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.076964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c55ad4-30c5-4d11-b67f-d6be42c72a56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.077665 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c55ad4-30c5-4d11-b67f-d6be42c72a56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:58.079784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c55ad4-30c5-4d11-b67f-d6be42c72a56 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:58.082698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 998/2001] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:58.091064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d875d9-a481-48e7-b7d4-5fa1290a7161 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:58.094282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-11d875d9-a481-48e7-b7d4-5fa1290a7161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:58.112279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d875d9-a481-48e7-b7d4-5fa1290a7161 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:58.117915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:58.118599 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.119922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.120330 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] PATH_INFO: `/v3/projects/b34f5f8ac78447fdbb185be350341193` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.122603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34f5f8ac78447fdbb185be350341193)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:58.125195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:58.148225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:58.207327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1004/2002] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:57 2022] POST /identity/v3/projects => generated 381 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:58.208279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c55ad4-30c5-4d11-b67f-d6be42c72a56 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:58.209855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-56c55ad4-30c5-4d11-b67f-d6be42c72a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:58.270613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c55ad4-30c5-4d11-b67f-d6be42c72a56 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:58.275096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34f5f8ac78447fdbb185be350341193', 'request_id': 'req-61eca3b5-008b-4ae9-90ce-6d8f38baf599'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:58.282667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34f5f8ac78447fdbb185be350341193', 'request_id': 'req-61eca3b5-008b-4ae9-90ce-6d8f38baf599'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:58.326545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f141e02-35ad-4c93-ad2e-5e9860eb87b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:58.327241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f141e02-35ad-4c93-ad2e-5e9860eb87b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.327550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f141e02-35ad-4c93-ad2e-5e9860eb87b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.327825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f141e02-35ad-4c93-ad2e-5e9860eb87b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.375448 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34f5f8ac78447fdbb185be350341193', 'request_id': 'req-61eca3b5-008b-4ae9-90ce-6d8f38baf599'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:58.398609 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f141e02-35ad-4c93-ad2e-5e9860eb87b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:58.400832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f141e02-35ad-4c93-ad2e-5e9860eb87b6 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:58.400997 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] The token cache is being invalidate because project b34f5f8ac78447fdbb185be350341193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:58.401207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:58.401962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:58.403908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 999/2003] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:58 2022] DELETE /identity/v3/projects/b34f5f8ac78447fdbb185be350341193 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:58.405012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:58.406771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:58.417209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1005/2004] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:42:58.418366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf42f58-e2dd-46ed-a8fa-a8162de709f4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.418694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf42f58-e2dd-46ed-a8fa-a8162de709f4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.418991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf42f58-e2dd-46ed-a8fa-a8162de709f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.433518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eca3b5-008b-4ae9-90ce-6d8f38baf599 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:58.441198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74471cc8-a9f7-4824-abca-8eddebea13c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:58.441783 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-eaf42f58-e2dd-46ed-a8fa-a8162de709f4 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:58.441961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74471cc8-a9f7-4824-abca-8eddebea13c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.442584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74471cc8-a9f7-4824-abca-8eddebea13c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.442988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74471cc8-a9f7-4824-abca-8eddebea13c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.443156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eaf42f58-e2dd-46ed-a8fa-a8162de709f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:58.444367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74471cc8-a9f7-4824-abca-8eddebea13c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:58.445232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:58.446645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74471cc8-a9f7-4824-abca-8eddebea13c5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:58.458934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1006/2005] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:58.472293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff3f0c-166a-4df5-9ac4-8f45fe9b604e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.472898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff3f0c-166a-4df5-9ac4-8f45fe9b604e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.475174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff3f0c-166a-4df5-9ac4-8f45fe9b604e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.497830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-69ff3f0c-166a-4df5-9ac4-8f45fe9b604e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:58.499790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-69ff3f0c-166a-4df5-9ac4-8f45fe9b604e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:58.511815 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1007/2006] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:58.513742 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff3f0c-166a-4df5-9ac4-8f45fe9b604e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:58.523685 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff3f0c-166a-4df5-9ac4-8f45fe9b604e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:58.528583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4c57e-dbf4-493c-9357-5eebcc8bc43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:58.533030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4c57e-dbf4-493c-9357-5eebcc8bc43f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.533030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4c57e-dbf4-493c-9357-5eebcc8bc43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.533030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4c57e-dbf4-493c-9357-5eebcc8bc43f admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.533030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4c57e-dbf4-493c-9357-5eebcc8bc43f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:42:58.533030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4c57e-dbf4-493c-9357-5eebcc8bc43f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:58.533555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4c57e-dbf4-493c-9357-5eebcc8bc43f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:58.550181 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1008/2007] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:42:58.551260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4c57e-dbf4-493c-9357-5eebcc8bc43f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:58.560318 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1000/2008] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:58 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:58.563567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4c57e-dbf4-493c-9357-5eebcc8bc43f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:58.569533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74471cc8-a9f7-4824-abca-8eddebea13c5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:58.570597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f2678-ba63-4f9f-b55b-a11507b0b49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:58.571343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f2678-ba63-4f9f-b55b-a11507b0b49e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.572367 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f2678-ba63-4f9f-b55b-a11507b0b49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.572915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f2678-ba63-4f9f-b55b-a11507b0b49e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.573856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f2678-ba63-4f9f-b55b-a11507b0b49e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:58.577711 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f2678-ba63-4f9f-b55b-a11507b0b49e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:58.580963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74471cc8-a9f7-4824-abca-8eddebea13c5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:58.588187 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1328d0-8390-4db5-9ff7-1ebb54a79215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:58.588828 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1328d0-8390-4db5-9ff7-1ebb54a79215 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.589113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1328d0-8390-4db5-9ff7-1ebb54a79215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.589395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1328d0-8390-4db5-9ff7-1ebb54a79215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.590104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1328d0-8390-4db5-9ff7-1ebb54a79215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:58.592425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1328d0-8390-4db5-9ff7-1ebb54a79215 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:58.604307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1001/2009] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:58.608735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1328d0-8390-4db5-9ff7-1ebb54a79215 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:58.620659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1328d0-8390-4db5-9ff7-1ebb54a79215 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:58.623471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacb050-6d16-4956-9857-926feebbc575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:58.624200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacb050-6d16-4956-9857-926feebbc575 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.624485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacb050-6d16-4956-9857-926feebbc575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.624757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacb050-6d16-4956-9857-926feebbc575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.625449 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacb050-6d16-4956-9857-926feebbc575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:58.627034 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacb050-6d16-4956-9857-926feebbc575 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:58.637457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1002/2010] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:58.641822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacb050-6d16-4956-9857-926feebbc575 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:58.652209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacb050-6d16-4956-9857-926feebbc575 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:58.657527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a360bd-c561-49d5-b880-35db086e18b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:58.658205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a360bd-c561-49d5-b880-35db086e18b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.658492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a360bd-c561-49d5-b880-35db086e18b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.658771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a360bd-c561-49d5-b880-35db086e18b7 admin admin] PATH_INFO: `/v3/projects/0ad5546f7a044937957b3551cbad8bfb/users/2cd827c8dfc54822b449992b2546923e/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.677481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a360bd-c561-49d5-b880-35db086e18b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad5546f7a044937957b3551cbad8bfb, user_id=2cd827c8dfc54822b449992b2546923e, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:58.680566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a360bd-c561-49d5-b880-35db086e18b7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:58.686300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1009/2011] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:58 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:58.690398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f2678-ba63-4f9f-b55b-a11507b0b49e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:58.773567 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1003/2012] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:58 2022] PUT /identity/v3/projects/0ad5546f7a044937957b3551cbad8bfb/users/2cd827c8dfc54822b449992b2546923e/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:58.774124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f2678-ba63-4f9f-b55b-a11507b0b49e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:58.776673 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd9044-2cd4-4e2a-9c5e-5e709e3f4a2c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.776971 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd9044-2cd4-4e2a-9c5e-5e709e3f4a2c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.777254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd9044-2cd4-4e2a-9c5e-5e709e3f4a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.815285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1dcd9044-2cd4-4e2a-9c5e-5e709e3f4a2c None None] MFA Rules not processed for user `2cd827c8dfc54822b449992b2546923e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:58.816947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1dcd9044-2cd4-4e2a-9c5e-5e709e3f4a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:58.860868 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e213-77cc-4140-9761-466fd46dc9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:58.862418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e213-77cc-4140-9761-466fd46dc9bf admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.862418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e213-77cc-4140-9761-466fd46dc9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.862418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e213-77cc-4140-9761-466fd46dc9bf admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.862904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e213-77cc-4140-9761-466fd46dc9bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:58.868292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e213-77cc-4140-9761-466fd46dc9bf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:58.911179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1004/2013] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:42:58.912217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd9044-2cd4-4e2a-9c5e-5e709e3f4a2c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:58.926876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd9044-2cd4-4e2a-9c5e-5e709e3f4a2c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:58.933107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accc93e-4c9e-4d17-8e58-ab7c0738a656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:58.936968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accc93e-4c9e-4d17-8e58-ab7c0738a656 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:58.936968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accc93e-4c9e-4d17-8e58-ab7c0738a656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:58.936968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accc93e-4c9e-4d17-8e58-ab7c0738a656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:58.936968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accc93e-4c9e-4d17-8e58-ab7c0738a656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:58.940771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accc93e-4c9e-4d17-8e58-ab7c0738a656 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:58.978660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1010/2014] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:58 2022] POST /identity/v3/users => generated 294 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:58.978660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e213-77cc-4140-9761-466fd46dc9bf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:58.979869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7bf2e213-77cc-4140-9761-466fd46dc9bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:58.996104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e213-77cc-4140-9761-466fd46dc9bf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:59.003506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:59.008255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.008255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.008255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] PATH_INFO: `/v3/projects/07715d4acc3c44548473850a510a8660` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.008255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07715d4acc3c44548473850a510a8660)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:59.008255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:59.080829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1005/2015] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:58 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:59.082140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accc93e-4c9e-4d17-8e58-ab7c0738a656 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:59.154341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accc93e-4c9e-4d17-8e58-ab7c0738a656 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:59.160560 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4cc8a-f231-4e7a-9f11-b1379c50a244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:59.161372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4cc8a-f231-4e7a-9f11-b1379c50a244 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.161929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4cc8a-f231-4e7a-9f11-b1379c50a244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.161929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4cc8a-f231-4e7a-9f11-b1379c50a244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.162680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4cc8a-f231-4e7a-9f11-b1379c50a244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:59.164302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4cc8a-f231-4e7a-9f11-b1379c50a244 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:59.172642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07715d4acc3c44548473850a510a8660', 'request_id': 'req-cef1072a-0084-4460-a8c9-758c36b21ec5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:59.176200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1006/2016] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:59.177951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4cc8a-f231-4e7a-9f11-b1379c50a244 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:59.186468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=79,keystone:DELETE=16,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:59.190511 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4cc8a-f231-4e7a-9f11-b1379c50a244 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:59.251923 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32850992-6526-443b-982a-3737aef8e914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:59.252614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32850992-6526-443b-982a-3737aef8e914 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.252910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32850992-6526-443b-982a-3737aef8e914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.253237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32850992-6526-443b-982a-3737aef8e914 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.254213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32850992-6526-443b-982a-3737aef8e914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:59.256988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32850992-6526-443b-982a-3737aef8e914 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:59.261818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07715d4acc3c44548473850a510a8660', 'request_id': 'req-cef1072a-0084-4460-a8c9-758c36b21ec5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:59.273605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07715d4acc3c44548473850a510a8660', 'request_id': 'req-cef1072a-0084-4460-a8c9-758c36b21ec5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:59.287967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] The token cache is being invalidate because project 07715d4acc3c44548473850a510a8660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:42:59.288440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:59.289215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:59.290814 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1011/2017] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:42:58 2022] DELETE /identity/v3/projects/07715d4acc3c44548473850a510a8660 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:59.291581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:59.293135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:59.311040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1072a-0084-4460-a8c9-758c36b21ec5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:59.316387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98106c5-7342-41de-b8bf-36bc51c1d3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:59.317216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98106c5-7342-41de-b8bf-36bc51c1d3fa admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.317570 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98106c5-7342-41de-b8bf-36bc51c1d3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.317923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98106c5-7342-41de-b8bf-36bc51c1d3fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.318700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98106c5-7342-41de-b8bf-36bc51c1d3fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:59.320582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98106c5-7342-41de-b8bf-36bc51c1d3fa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:59.333445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1012/2018] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:59.333445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98106c5-7342-41de-b8bf-36bc51c1d3fa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:59.333445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f98106c5-7342-41de-b8bf-36bc51c1d3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:59.347481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:59.352033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98106c5-7342-41de-b8bf-36bc51c1d3fa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:59.356925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:59.358323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.358710 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.359189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] PATH_INFO: `/v3/users/2c0602637b5449e9905d429bc26f02e9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.361758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0602637b5449e9905d429bc26f02e9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:59.364583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:59.381878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1007/2019] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:59 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:59.382708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32850992-6526-443b-982a-3737aef8e914 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:59.397177 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32850992-6526-443b-982a-3737aef8e914 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:59.402344 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58288c0-3820-4556-a825-054d72d86008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:59.403175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58288c0-3820-4556-a825-054d72d86008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.403472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58288c0-3820-4556-a825-054d72d86008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.403751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58288c0-3820-4556-a825-054d72d86008 admin admin] PATH_INFO: `/v3/projects/6051d2259fb34f0292d82ba737345069/users/8b6ddc4823424e37b1e13879d970ad62/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.421794 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58288c0-3820-4556-a825-054d72d86008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6051d2259fb34f0292d82ba737345069, user_id=8b6ddc4823424e37b1e13879d970ad62, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:59.425966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58288c0-3820-4556-a825-054d72d86008 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:59.496127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1008/2020] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:59 2022] PUT /identity/v3/projects/6051d2259fb34f0292d82ba737345069/users/8b6ddc4823424e37b1e13879d970ad62/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:59.496579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58288c0-3820-4556-a825-054d72d86008 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:59.498883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d58288c0-3820-4556-a825-054d72d86008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:59.555794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0602637b5449e9905d429bc26f02e9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:59.557323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0602637b5449e9905d429bc26f02e9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:59.566411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0602637b5449e9905d429bc26f02e9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:59.581283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0602637b5449e9905d429bc26f02e9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:59.593621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58288c0-3820-4556-a825-054d72d86008 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:59.598887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62647029-c970-47c3-8c35-0fc5af944591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:59.600695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62647029-c970-47c3-8c35-0fc5af944591 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.600695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62647029-c970-47c3-8c35-0fc5af944591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.600695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62647029-c970-47c3-8c35-0fc5af944591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.600911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62647029-c970-47c3-8c35-0fc5af944591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:59.602679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62647029-c970-47c3-8c35-0fc5af944591 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:59.618941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1009/2021] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:42:59.618941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf88940-e3e0-4e48-ac46-9eeef0837312 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.618941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf88940-e3e0-4e48-ac46-9eeef0837312 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.618941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf88940-e3e0-4e48-ac46-9eeef0837312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.647151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-acf88940-e3e0-4e48-ac46-9eeef0837312 None None] MFA Rules not processed for user `8b6ddc4823424e37b1e13879d970ad62`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:42:59.648428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-acf88940-e3e0-4e48-ac46-9eeef0837312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:59.685259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0602637b5449e9905d429bc26f02e9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:42:59.696040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1013/2022] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:42:59 2022] DELETE /identity/v3/users/2c0602637b5449e9905d429bc26f02e9 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:59.696040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:59.696040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:59.741292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1010/2023] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:42:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:42:59.743054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf88940-e3e0-4e48-ac46-9eeef0837312 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:59.745204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-acf88940-e3e0-4e48-ac46-9eeef0837312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:42:59.765805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf88940-e3e0-4e48-ac46-9eeef0837312 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:59.771111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf1d4e-42bf-458f-b760-06b093fbafd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:59.771762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf1d4e-42bf-458f-b760-06b093fbafd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.772075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf1d4e-42bf-458f-b760-06b093fbafd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.772409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf1d4e-42bf-458f-b760-06b093fbafd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.772695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7123722-dd09-4040-baef-b2bbbc919056 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:59.773092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf1d4e-42bf-458f-b760-06b093fbafd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:59.775000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf1d4e-42bf-458f-b760-06b093fbafd4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:59.778153 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd768a94-2d31-4a06-b481-87f3763b0869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:59.778939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd768a94-2d31-4a06-b481-87f3763b0869 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.779338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd768a94-2d31-4a06-b481-87f3763b0869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.779709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd768a94-2d31-4a06-b481-87f3763b0869 admin admin] PATH_INFO: `/v3/projects/26d217c7aed04662ba6f8f1d52fea915/users/f573600267b04f089dc8878982f1f657/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.796684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd768a94-2d31-4a06-b481-87f3763b0869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d217c7aed04662ba6f8f1d52fea915, user_id=f573600267b04f089dc8878982f1f657, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:59.799556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd768a94-2d31-4a06-b481-87f3763b0869 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:59.866307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1014/2024] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:59 2022] PUT /identity/v3/projects/26d217c7aed04662ba6f8f1d52fea915/users/f573600267b04f089dc8878982f1f657/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:42:59.870367 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd768a94-2d31-4a06-b481-87f3763b0869 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:59.875348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1011/2025] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:42:59 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:42:59.876380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf1d4e-42bf-458f-b760-06b093fbafd4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:42:59.913844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:42:59.946707 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd768a94-2d31-4a06-b481-87f3763b0869 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:59.954093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c85d02-67fb-4793-b5e4-ad5c0cf2c6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:59.955730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c85d02-67fb-4793-b5e4-ad5c0cf2c6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.956128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c85d02-67fb-4793-b5e4-ad5c0cf2c6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.956946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c85d02-67fb-4793-b5e4-ad5c0cf2c6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.957994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c85d02-67fb-4793-b5e4-ad5c0cf2c6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:59.959609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf1d4e-42bf-458f-b760-06b093fbafd4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:42:59.960439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c85d02-67fb-4793-b5e4-ad5c0cf2c6c5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:59.964773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad3be2-93db-4000-bf99-01428b5f62df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:42:59.965720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad3be2-93db-4000-bf99-01428b5f62df admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:42:59.966462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad3be2-93db-4000-bf99-01428b5f62df admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:42:59.967884 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad3be2-93db-4000-bf99-01428b5f62df admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:42:59.968861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad3be2-93db-4000-bf99-01428b5f62df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:42:59.971684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad3be2-93db-4000-bf99-01428b5f62df admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:42:59.977355 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1015/2026] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:42:59.983706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c85d02-67fb-4793-b5e4-ad5c0cf2c6c5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:00.002694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c85d02-67fb-4793-b5e4-ad5c0cf2c6c5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:00.007803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afc2a3-8de4-4cf5-b6e1-143bd91cae78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:00.008523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afc2a3-8de4-4cf5-b6e1-143bd91cae78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.008816 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afc2a3-8de4-4cf5-b6e1-143bd91cae78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.009098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afc2a3-8de4-4cf5-b6e1-143bd91cae78 admin admin] PATH_INFO: `/v3/projects/26d217c7aed04662ba6f8f1d52fea915/users/f573600267b04f089dc8878982f1f657/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.011223 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afc2a3-8de4-4cf5-b6e1-143bd91cae78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d217c7aed04662ba6f8f1d52fea915, user_id=f573600267b04f089dc8878982f1f657, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:00.014084 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afc2a3-8de4-4cf5-b6e1-143bd91cae78 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:00.029630 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1016/2027] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:42:59 2022] PUT /identity/v3/projects/26d217c7aed04662ba6f8f1d52fea915/users/f573600267b04f089dc8878982f1f657/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:43:00.038317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f7db1-a745-4f8d-8356-fab25020ced8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.038616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f7db1-a745-4f8d-8356-fab25020ced8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.038900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f7db1-a745-4f8d-8356-fab25020ced8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.070757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3e6f7db1-a745-4f8d-8356-fab25020ced8 None None] MFA Rules not processed for user `f573600267b04f089dc8878982f1f657`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:00.072093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3e6f7db1-a745-4f8d-8356-fab25020ced8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:00.090682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1012/2028] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:42:59 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:00.095180 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad3be2-93db-4000-bf99-01428b5f62df admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:00.110703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad3be2-93db-4000-bf99-01428b5f62df admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:00.174225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:00.181782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1017/2029] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:00.193656 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f7db1-a745-4f8d-8356-fab25020ced8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:00.195608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7253e0-650c-4643-8670-124ffaa96622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:00.195963 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3e6f7db1-a745-4f8d-8356-fab25020ced8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:00.198282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7253e0-650c-4643-8670-124ffaa96622 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.202147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7253e0-650c-4643-8670-124ffaa96622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.202147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7253e0-650c-4643-8670-124ffaa96622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.202147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7253e0-650c-4643-8670-124ffaa96622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:00.204175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7253e0-650c-4643-8670-124ffaa96622 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:00.215743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1013/2030] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:43:00.221113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7253e0-650c-4643-8670-124ffaa96622 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:00.238206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7253e0-650c-4643-8670-124ffaa96622 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:00.252056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd3702-0937-491c-a85c-725eb2d6ba44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:00.252859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd3702-0937-491c-a85c-725eb2d6ba44 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.253152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd3702-0937-491c-a85c-725eb2d6ba44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.253581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd3702-0937-491c-a85c-725eb2d6ba44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.254305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd3702-0937-491c-a85c-725eb2d6ba44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:00.257620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd3702-0937-491c-a85c-725eb2d6ba44 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:00.287948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1014/2031] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:00.289699 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd3702-0937-491c-a85c-725eb2d6ba44 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:00.302231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd3702-0937-491c-a85c-725eb2d6ba44 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:00.307574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b024b-c55e-46b2-b3b6-e730e00d17b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:00.308309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b024b-c55e-46b2-b3b6-e730e00d17b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.308712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b024b-c55e-46b2-b3b6-e730e00d17b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.309013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b024b-c55e-46b2-b3b6-e730e00d17b6 admin admin] PATH_INFO: `/v3/projects/88c8eb581d2c46c8a006effaf641e034/users/d60ed3a7fdb04887bc801c213ec47440/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.321228 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f7db1-a745-4f8d-8356-fab25020ced8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:00.326972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b024b-c55e-46b2-b3b6-e730e00d17b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c8eb581d2c46c8a006effaf641e034, user_id=d60ed3a7fdb04887bc801c213ec47440, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:00.330194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4016d-a598-4593-9cf3-942d9a5de6ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:00.332836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b024b-c55e-46b2-b3b6-e730e00d17b6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:00.332949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4016d-a598-4593-9cf3-942d9a5de6ea service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.332949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4016d-a598-4593-9cf3-942d9a5de6ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.332949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4016d-a598-4593-9cf3-942d9a5de6ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.343948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4016d-a598-4593-9cf3-942d9a5de6ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:00.345490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4016d-a598-4593-9cf3-942d9a5de6ea service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:00.355803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1018/2032] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:00.356727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4016d-a598-4593-9cf3-942d9a5de6ea service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:00.359503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-00a4016d-a598-4593-9cf3-942d9a5de6ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:00.374399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4016d-a598-4593-9cf3-942d9a5de6ea service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:00.379833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:00.380583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.380872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.381184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] PATH_INFO: `/v3/users/92fa59772a574f42b96a0776c269b93c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.382359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=92fa59772a574f42b96a0776c269b93c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:00.384436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:00.425720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1015/2033] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:00 2022] PUT /identity/v3/projects/88c8eb581d2c46c8a006effaf641e034/users/d60ed3a7fdb04887bc801c213ec47440/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:00.435790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ac166-a947-468b-9523-3f37c5672d88 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.436120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ac166-a947-468b-9523-3f37c5672d88 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.436406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ac166-a947-468b-9523-3f37c5672d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.463606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3c3ac166-a947-468b-9523-3f37c5672d88 None None] MFA Rules not processed for user `d60ed3a7fdb04887bc801c213ec47440`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:00.464837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3c3ac166-a947-468b-9523-3f37c5672d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:00.550152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1016/2034] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:00.550152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ac166-a947-468b-9523-3f37c5672d88 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:00.550152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3c3ac166-a947-468b-9523-3f37c5672d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:00.604602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92fa59772a574f42b96a0776c269b93c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:00.605345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92fa59772a574f42b96a0776c269b93c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:00.613658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92fa59772a574f42b96a0776c269b93c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:00.628743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ac166-a947-468b-9523-3f37c5672d88 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:00.629366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92fa59772a574f42b96a0776c269b93c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:00.634431 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:00.635083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.635368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.635720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] PATH_INFO: `/v3/projects/397f7a4ce45048618e6c46d1becd8744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.637246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=397f7a4ce45048618e6c46d1becd8744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:00.640192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:00.693972 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92fa59772a574f42b96a0776c269b93c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:00.700412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1019/2035] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:00 2022] DELETE /identity/v3/users/92fa59772a574f42b96a0776c269b93c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:00.701834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:00.703388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:00.718253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:00.767032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '397f7a4ce45048618e6c46d1becd8744', 'request_id': 'req-fef0db84-4c3e-43d8-af60-d5b223b95c11'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:00.775912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '397f7a4ce45048618e6c46d1becd8744', 'request_id': 'req-fef0db84-4c3e-43d8-af60-d5b223b95c11'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:00.796094 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2517b-a3ce-4457-84b3-2a118c1075fd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:00.811911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:00.857381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '397f7a4ce45048618e6c46d1becd8744', 'request_id': 'req-fef0db84-4c3e-43d8-af60-d5b223b95c11'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:00.871861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a2d31-ba4a-4989-ba47-ce22ece3d814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:00.872306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] The token cache is being invalidate because project 397f7a4ce45048618e6c46d1becd8744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:00.872605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:00.872735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a2d31-ba4a-4989-ba47-ce22ece3d814 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.874227 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fef0db84-4c3e-43d8-af60-d5b223b95c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:00.874691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a2d31-ba4a-4989-ba47-ce22ece3d814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.875046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a2d31-ba4a-4989-ba47-ce22ece3d814 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.875874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a2d31-ba4a-4989-ba47-ce22ece3d814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:00.877943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a2d31-ba4a-4989-ba47-ce22ece3d814 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:00.883495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1017/2036] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:00 2022] DELETE /identity/v3/projects/397f7a4ce45048618e6c46d1becd8744 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:00.888168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baf823-d9b8-4bda-baeb-97211227b6ea None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.889220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baf823-d9b8-4bda-baeb-97211227b6ea None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.889376 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baf823-d9b8-4bda-baeb-97211227b6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.909788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-82baf823-d9b8-4bda-baeb-97211227b6ea None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:00.911002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-82baf823-d9b8-4bda-baeb-97211227b6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:00.919239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1018/2037] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:00.927136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dc57a-64fb-40d9-b90e-f77fbfe4ec2e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.927425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dc57a-64fb-40d9-b90e-f77fbfe4ec2e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.927705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dc57a-64fb-40d9-b90e-f77fbfe4ec2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.947328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-303dc57a-64fb-40d9-b90e-f77fbfe4ec2e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:00.948541 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-303dc57a-64fb-40d9-b90e-f77fbfe4ec2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:00.958214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1019/2038] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:00.962976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303dc57a-64fb-40d9-b90e-f77fbfe4ec2e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:00.972705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303dc57a-64fb-40d9-b90e-f77fbfe4ec2e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:00.980279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15b8c0-e281-447c-b357-62a41097c858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:00.981083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b8c0-e281-447c-b357-62a41097c858 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:00.981414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b8c0-e281-447c-b357-62a41097c858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:00.982928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b8c0-e281-447c-b357-62a41097c858 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:00.982928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15b8c0-e281-447c-b357-62a41097c858 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:43:00.982928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15b8c0-e281-447c-b357-62a41097c858 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:00.984665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15b8c0-e281-447c-b357-62a41097c858 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:00.991449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1020/2039] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:00 2022] POST /identity/v3/projects => generated 387 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:00.991449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a2d31-ba4a-4989-ba47-ce22ece3d814 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:00.993205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9f5a2d31-ba4a-4989-ba47-ce22ece3d814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:00.998893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1020/2040] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:43:00.999822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15b8c0-e281-447c-b357-62a41097c858 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:01.017272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fe15b8c0-e281-447c-b357-62a41097c858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:01.017272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15b8c0-e281-447c-b357-62a41097c858 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:01.063517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a2d31-ba4a-4989-ba47-ce22ece3d814 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:01.068158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:01.068794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.069075 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.070741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] PATH_INFO: `/v3/users/2cd827c8dfc54822b449992b2546923e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.070741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd827c8dfc54822b449992b2546923e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:01.072999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:01.077448 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:01.078096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.078382 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.079317 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] PATH_INFO: `/v3/projects/5dc837002aaa47979fe783bc0bc1cbf2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.080539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc837002aaa47979fe783bc0bc1cbf2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:01.082813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:01.106014 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:01.275914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd827c8dfc54822b449992b2546923e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:01.276399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc837002aaa47979fe783bc0bc1cbf2', 'request_id': 'req-756756dd-0fd9-403b-854a-a2b26b187f8f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:01.276596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd827c8dfc54822b449992b2546923e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:01.282642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc837002aaa47979fe783bc0bc1cbf2', 'request_id': 'req-756756dd-0fd9-403b-854a-a2b26b187f8f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:01.283498 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd827c8dfc54822b449992b2546923e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:01.302868 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd827c8dfc54822b449992b2546923e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:01.381919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc837002aaa47979fe783bc0bc1cbf2', 'request_id': 'req-756756dd-0fd9-403b-854a-a2b26b187f8f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:01.391638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:01.392862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] The token cache is being invalidate because project 5dc837002aaa47979fe783bc0bc1cbf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:01.393157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:01.393897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:01.395525 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1021/2041] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:00 2022] DELETE /identity/v3/projects/5dc837002aaa47979fe783bc0bc1cbf2 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:01.399569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:01.401002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:01.444767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd827c8dfc54822b449992b2546923e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:01.453158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1021/2042] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:00 2022] DELETE /identity/v3/users/2cd827c8dfc54822b449992b2546923e => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:01.454015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:01.455335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:01.469312 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756756dd-0fd9-403b-854a-a2b26b187f8f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:01.532693 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd0696-bd5c-4350-afdb-b040b532f84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:01.533327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd0696-bd5c-4350-afdb-b040b532f84f admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.533615 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd0696-bd5c-4350-afdb-b040b532f84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.533715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af60f7-db48-4f96-9782-b327f564ef33 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:01.533894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd0696-bd5c-4350-afdb-b040b532f84f admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.534756 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd0696-bd5c-4350-afdb-b040b532f84f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:01.536731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd0696-bd5c-4350-afdb-b040b532f84f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:01.538673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5db16f-11b7-44f6-b8ec-df356ae8ae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:01.541010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5db16f-11b7-44f6-b8ec-df356ae8ae60 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.541010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5db16f-11b7-44f6-b8ec-df356ae8ae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.541010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5db16f-11b7-44f6-b8ec-df356ae8ae60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.541010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5db16f-11b7-44f6-b8ec-df356ae8ae60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:01.542674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5db16f-11b7-44f6-b8ec-df356ae8ae60 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:01.543255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:01.614210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1022/2043] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:01 2022] POST /identity/v3/users => generated 300 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:01.614210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d4269-a0a0-47b9-92c3-c5670e75b8be None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.614210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d4269-a0a0-47b9-92c3-c5670e75b8be None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.614210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d4269-a0a0-47b9-92c3-c5670e75b8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.632440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2c9d4269-a0a0-47b9-92c3-c5670e75b8be None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:01.637209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2c9d4269-a0a0-47b9-92c3-c5670e75b8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:01.637327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1022/2044] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:01 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:01.637327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562a05e-bdce-407e-9c20-f2d525de58bf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.637327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562a05e-bdce-407e-9c20-f2d525de58bf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.637327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562a05e-bdce-407e-9c20-f2d525de58bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.657061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1562a05e-bdce-407e-9c20-f2d525de58bf None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:01.658311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1562a05e-bdce-407e-9c20-f2d525de58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:01.694674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1023/2045] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:01.695474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d4269-a0a0-47b9-92c3-c5670e75b8be None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:01.710353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d4269-a0a0-47b9-92c3-c5670e75b8be None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:01.715295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9727e-89d6-4a21-9e94-60e25e844332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:01.715916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9727e-89d6-4a21-9e94-60e25e844332 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.716233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9727e-89d6-4a21-9e94-60e25e844332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.717043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9727e-89d6-4a21-9e94-60e25e844332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.717742 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9727e-89d6-4a21-9e94-60e25e844332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:01.719375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9727e-89d6-4a21-9e94-60e25e844332 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:01.723486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1023/2046] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:01.723486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562a05e-bdce-407e-9c20-f2d525de58bf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:01.723486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1562a05e-bdce-407e-9c20-f2d525de58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:01.729826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1024/2047] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:01.731297 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4040ce-901c-4a8f-b489-d621e2bd4454 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.731593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4040ce-901c-4a8f-b489-d621e2bd4454 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.731930 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4040ce-901c-4a8f-b489-d621e2bd4454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.739252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562a05e-bdce-407e-9c20-f2d525de58bf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:01.744191 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb89e83-61b3-4c4d-af6f-a24bba80b7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:01.744813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb89e83-61b3-4c4d-af6f-a24bba80b7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.745090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb89e83-61b3-4c4d-af6f-a24bba80b7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.745363 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb89e83-61b3-4c4d-af6f-a24bba80b7d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.746092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb89e83-61b3-4c4d-af6f-a24bba80b7d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:01.748080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb89e83-61b3-4c4d-af6f-a24bba80b7d8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:01.752426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9c4040ce-901c-4a8f-b489-d621e2bd4454 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:01.753691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9c4040ce-901c-4a8f-b489-d621e2bd4454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:01.761974 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1025/2048] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:01.762823 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4040ce-901c-4a8f-b489-d621e2bd4454 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:01.764179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9c4040ce-901c-4a8f-b489-d621e2bd4454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:01.822978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4040ce-901c-4a8f-b489-d621e2bd4454 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:01.826858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0d8cc-959b-46cc-88c8-4655dd2cf892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:01.827473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0d8cc-959b-46cc-88c8-4655dd2cf892 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.827759 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0d8cc-959b-46cc-88c8-4655dd2cf892 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.828065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0d8cc-959b-46cc-88c8-4655dd2cf892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.837265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0d8cc-959b-46cc-88c8-4655dd2cf892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:01.838688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0d8cc-959b-46cc-88c8-4655dd2cf892 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:01.848983 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1026/2049] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:01.849921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0d8cc-959b-46cc-88c8-4655dd2cf892 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:01.853114 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1024/2050] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:01 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:01.853940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb89e83-61b3-4c4d-af6f-a24bba80b7d8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:01.860532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0d8cc-959b-46cc-88c8-4655dd2cf892 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:01.865520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb89e83-61b3-4c4d-af6f-a24bba80b7d8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:01.865620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec66f-866f-4574-98b7-9722ecd53d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:01.866390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec66f-866f-4574-98b7-9722ecd53d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.866514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec66f-866f-4574-98b7-9722ecd53d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.866867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec66f-866f-4574-98b7-9722ecd53d5a admin admin] PATH_INFO: `/v3/projects/ec2272c1d63f40a09baa16853c6a84dc/users/16eea5863af3457e92903b620378edeb/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.870844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b35cb-a356-41de-88a5-a342ba4f80dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:01.871775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b35cb-a356-41de-88a5-a342ba4f80dd admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.873543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:01.874277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b35cb-a356-41de-88a5-a342ba4f80dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.874719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b35cb-a356-41de-88a5-a342ba4f80dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.875607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b35cb-a356-41de-88a5-a342ba4f80dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:43:01.876050 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b35cb-a356-41de-88a5-a342ba4f80dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:01.877840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b35cb-a356-41de-88a5-a342ba4f80dd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:01.883810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ec66f-866f-4574-98b7-9722ecd53d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2272c1d63f40a09baa16853c6a84dc, user_id=16eea5863af3457e92903b620378edeb, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:01.886526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ec66f-866f-4574-98b7-9722ecd53d5a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:01.892147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1025/2051] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:43:01.892888 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b35cb-a356-41de-88a5-a342ba4f80dd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:01.902820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b35cb-a356-41de-88a5-a342ba4f80dd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:01.908660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a613-25b9-45d5-a9fe-f0d750507b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:01.909270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a613-25b9-45d5-a9fe-f0d750507b89 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.909581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a613-25b9-45d5-a9fe-f0d750507b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.909859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a613-25b9-45d5-a9fe-f0d750507b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.910573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2a613-25b9-45d5-a9fe-f0d750507b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:01.915835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2a613-25b9-45d5-a9fe-f0d750507b89 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:01.922706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1026/2052] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:01.923511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a613-25b9-45d5-a9fe-f0d750507b89 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:01.935010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a613-25b9-45d5-a9fe-f0d750507b89 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:01.940839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4b212-b10a-4c67-805e-44271c3008ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:01.941461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4b212-b10a-4c67-805e-44271c3008ef admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:01.941750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4b212-b10a-4c67-805e-44271c3008ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:01.942032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4b212-b10a-4c67-805e-44271c3008ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:01.942187 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:01.942713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4b212-b10a-4c67-805e-44271c3008ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:01.944697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4b212-b10a-4c67-805e-44271c3008ef admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:01.976426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1027/2053] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:01 2022] PUT /identity/v3/projects/ec2272c1d63f40a09baa16853c6a84dc/users/16eea5863af3457e92903b620378edeb/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:01.977251 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec66f-866f-4574-98b7-9722ecd53d5a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.037988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec66f-866f-4574-98b7-9722ecd53d5a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.042622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b866e9b-90f6-400f-8605-247749f875ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.043269 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b866e9b-90f6-400f-8605-247749f875ed admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.043557 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b866e9b-90f6-400f-8605-247749f875ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.043833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b866e9b-90f6-400f-8605-247749f875ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.044630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b866e9b-90f6-400f-8605-247749f875ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.046785 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b866e9b-90f6-400f-8605-247749f875ed admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:02.056543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1028/2054] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:02.057322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b866e9b-90f6-400f-8605-247749f875ed admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.069568 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b866e9b-90f6-400f-8605-247749f875ed admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.073769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1027/2055] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:01 2022] POST /identity/v3/projects => generated 389 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:02.073769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4b212-b10a-4c67-805e-44271c3008ef admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.136907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d5495-e9ae-46f0-8818-6f44ed3e7020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.137720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d5495-e9ae-46f0-8818-6f44ed3e7020 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.138003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d5495-e9ae-46f0-8818-6f44ed3e7020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.138278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d5495-e9ae-46f0-8818-6f44ed3e7020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.139020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d5495-e9ae-46f0-8818-6f44ed3e7020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.141218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d5495-e9ae-46f0-8818-6f44ed3e7020 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:02.147453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4b212-b10a-4c67-805e-44271c3008ef admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.151564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1029/2056] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:02.152387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d5495-e9ae-46f0-8818-6f44ed3e7020 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.156307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3c4fb-a3f7-4fe1-9883-e3247ac070db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.157006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3c4fb-a3f7-4fe1-9883-e3247ac070db admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.157293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3c4fb-a3f7-4fe1-9883-e3247ac070db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.157633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3c4fb-a3f7-4fe1-9883-e3247ac070db admin admin] PATH_INFO: `/v3/projects/08c8e191d1bb4d1ea39cbf1c2eead475/users/c710058b96cb424dbb5b94d2c5d0f278/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.163672 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d5495-e9ae-46f0-8818-6f44ed3e7020 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.168948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911930e-2930-4223-9cd6-ee75e97090ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.170221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911930e-2930-4223-9cd6-ee75e97090ff admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.171110 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911930e-2930-4223-9cd6-ee75e97090ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.172160 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911930e-2930-4223-9cd6-ee75e97090ff admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.172921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911930e-2930-4223-9cd6-ee75e97090ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.174714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3c4fb-a3f7-4fe1-9883-e3247ac070db admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c8e191d1bb4d1ea39cbf1c2eead475, user_id=c710058b96cb424dbb5b94d2c5d0f278, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.174834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911930e-2930-4223-9cd6-ee75e97090ff admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:02.177563 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3c4fb-a3f7-4fe1-9883-e3247ac070db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:02.199020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=2,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:02.254797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1028/2057] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:02 2022] PUT /identity/v3/projects/08c8e191d1bb4d1ea39cbf1c2eead475/users/c710058b96cb424dbb5b94d2c5d0f278/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:02.255597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3c4fb-a3f7-4fe1-9883-e3247ac070db admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.315039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:02.317022 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3c4fb-a3f7-4fe1-9883-e3247ac070db admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.321135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1030/2058] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:02 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:02.321976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2993e3-9264-4129-ae69-d64476e12a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.322390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7be4e1-3525-45c8-a491-854d50e9017c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.322680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7be4e1-3525-45c8-a491-854d50e9017c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.322795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2993e3-9264-4129-ae69-d64476e12a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.323011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7be4e1-3525-45c8-a491-854d50e9017c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.323124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2993e3-9264-4129-ae69-d64476e12a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.323298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2993e3-9264-4129-ae69-d64476e12a76 admin admin] PATH_INFO: `/v3/projects/ec2272c1d63f40a09baa16853c6a84dc/users/16eea5863af3457e92903b620378edeb/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.325390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2993e3-9264-4129-ae69-d64476e12a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2272c1d63f40a09baa16853c6a84dc, user_id=16eea5863af3457e92903b620378edeb, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.328113 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2993e3-9264-4129-ae69-d64476e12a76 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:02.348323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1029/2059] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:02 2022] PUT /identity/v3/projects/ec2272c1d63f40a09baa16853c6a84dc/users/16eea5863af3457e92903b620378edeb/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:02.348323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2993e3-9264-4129-ae69-d64476e12a76 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.368016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fb7be4e1-3525-45c8-a491-854d50e9017c None None] MFA Rules not processed for user `c710058b96cb424dbb5b94d2c5d0f278`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:02.369798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fb7be4e1-3525-45c8-a491-854d50e9017c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:02.426993 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2993e3-9264-4129-ae69-d64476e12a76 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.434187 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d32641-b11d-46b6-b157-582f6f597f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.434919 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32641-b11d-46b6-b157-582f6f597f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.435210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32641-b11d-46b6-b157-582f6f597f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.436107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32641-b11d-46b6-b157-582f6f597f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.436573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d32641-b11d-46b6-b157-582f6f597f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.438447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d32641-b11d-46b6-b157-582f6f597f2c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:02.449408 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1030/2060] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:02.450116 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d32641-b11d-46b6-b157-582f6f597f2c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.453967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e1d32641-b11d-46b6-b157-582f6f597f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:02.467743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d32641-b11d-46b6-b157-582f6f597f2c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.472070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1031/2061] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:02.472558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.473199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.473395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53679387-2106-4651-bc9d-8eb7a7786575 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.473491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.473691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53679387-2106-4651-bc9d-8eb7a7786575 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.473797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] PATH_INFO: `/v3/projects/0ad5546f7a044937957b3551cbad8bfb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.473971 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53679387-2106-4651-bc9d-8eb7a7786575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.474977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad5546f7a044937957b3551cbad8bfb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.477036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:02.501485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-53679387-2106-4651-bc9d-8eb7a7786575 None None] MFA Rules not processed for user `16eea5863af3457e92903b620378edeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:02.502726 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-53679387-2106-4651-bc9d-8eb7a7786575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:02.578762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1032/2062] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:02.578762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53679387-2106-4651-bc9d-8eb7a7786575 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.590408 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53679387-2106-4651-bc9d-8eb7a7786575 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.596850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e867aa1-e981-4470-96c2-ceeeb106a5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.598116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e867aa1-e981-4470-96c2-ceeeb106a5e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.598116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e867aa1-e981-4470-96c2-ceeeb106a5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.598291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e867aa1-e981-4470-96c2-ceeeb106a5e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.598999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e867aa1-e981-4470-96c2-ceeeb106a5e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.600806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e867aa1-e981-4470-96c2-ceeeb106a5e5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:02.616290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1033/2063] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:02.616290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e867aa1-e981-4470-96c2-ceeeb106a5e5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.626739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad5546f7a044937957b3551cbad8bfb', 'request_id': 'req-cdd86fef-eea0-4a80-a6a3-e90a802caee2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:02.629823 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e867aa1-e981-4470-96c2-ceeeb106a5e5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.636704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:02.695923 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c188a9-0f92-4287-bdce-c107e3d82858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.696764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c188a9-0f92-4287-bdce-c107e3d82858 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.697211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c188a9-0f92-4287-bdce-c107e3d82858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.697493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c188a9-0f92-4287-bdce-c107e3d82858 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.698165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c188a9-0f92-4287-bdce-c107e3d82858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.700104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c188a9-0f92-4287-bdce-c107e3d82858 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:02.719585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad5546f7a044937957b3551cbad8bfb', 'request_id': 'req-cdd86fef-eea0-4a80-a6a3-e90a802caee2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:02.721973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:02.727021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad5546f7a044937957b3551cbad8bfb', 'request_id': 'req-cdd86fef-eea0-4a80-a6a3-e90a802caee2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:02.738337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] The token cache is being invalidate because project 0ad5546f7a044937957b3551cbad8bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:02.738624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:02.739265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:02.740875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1031/2064] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:02 2022] DELETE /identity/v3/projects/0ad5546f7a044937957b3551cbad8bfb => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:02.741632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.742888 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:02.798314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd86fef-eea0-4a80-a6a3-e90a802caee2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.805522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e029361-951f-430d-84c9-332c734952bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.806128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e029361-951f-430d-84c9-332c734952bd service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.806414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e029361-951f-430d-84c9-332c734952bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.806690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e029361-951f-430d-84c9-332c734952bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.808711 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1e029361-951f-430d-84c9-332c734952bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:02.810929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1034/2065] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:02 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:02.812566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c188a9-0f92-4287-bdce-c107e3d82858 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.814237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f3c188a9-0f92-4287-bdce-c107e3d82858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:02.881691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c188a9-0f92-4287-bdce-c107e3d82858 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.886576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41a152-8fdf-4d88-b69e-0c97b25825a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.887234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41a152-8fdf-4d88-b69e-0c97b25825a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.887523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41a152-8fdf-4d88-b69e-0c97b25825a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.887802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41a152-8fdf-4d88-b69e-0c97b25825a6 admin admin] PATH_INFO: `/v3/projects/ba45e1e6970248b192b9d665d4e45d1f/users/e5fe46987d5942b3a0ec5495aa0040ac/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.891067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e029361-951f-430d-84c9-332c734952bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.892551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e029361-951f-430d-84c9-332c734952bd service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:02.902435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1032/2066] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:02.903244 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e029361-951f-430d-84c9-332c734952bd service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:02.905137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41a152-8fdf-4d88-b69e-0c97b25825a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba45e1e6970248b192b9d665d4e45d1f, user_id=e5fe46987d5942b3a0ec5495aa0040ac, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.905345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1e029361-951f-430d-84c9-332c734952bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:02.908233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41a152-8fdf-4d88-b69e-0c97b25825a6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:02.918559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e029361-951f-430d-84c9-332c734952bd service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:02.924151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:02.924803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:02.925077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:02.925358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] PATH_INFO: `/v3/users/f573600267b04f089dc8878982f1f657` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:02.926540 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f573600267b04f089dc8878982f1f657)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:02.928497 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:03.028494 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1035/2067] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:02 2022] PUT /identity/v3/projects/ba45e1e6970248b192b9d665d4e45d1f/users/e5fe46987d5942b3a0ec5495aa0040ac/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:03.029444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41a152-8fdf-4d88-b69e-0c97b25825a6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:03.032165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fd41a152-8fdf-4d88-b69e-0c97b25825a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:03.082433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:03.102109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41a152-8fdf-4d88-b69e-0c97b25825a6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:03.107003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6941ba7-f92b-4fdf-a0f8-466660700035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:03.108345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6941ba7-f92b-4fdf-a0f8-466660700035 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:03.108640 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6941ba7-f92b-4fdf-a0f8-466660700035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:03.108854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:03.108983 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6941ba7-f92b-4fdf-a0f8-466660700035 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:03.109570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6941ba7-f92b-4fdf-a0f8-466660700035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:03.111294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f573600267b04f089dc8878982f1f657'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:03.111670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6941ba7-f92b-4fdf-a0f8-466660700035 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:03.111971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f573600267b04f089dc8878982f1f657'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:03.119045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f573600267b04f089dc8878982f1f657'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:03.134346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f573600267b04f089dc8878982f1f657'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:03.238874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f573600267b04f089dc8878982f1f657'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:03.245402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1036/2068] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:03 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:03.245618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1033/2069] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:02 2022] DELETE /identity/v3/users/f573600267b04f089dc8878982f1f657 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:03.249961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:03.250232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce24afd-412e-4716-8b09-3c529f6e72e3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:03.250872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce24afd-412e-4716-8b09-3c529f6e72e3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:03.251512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce24afd-412e-4716-8b09-3c529f6e72e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:03.252831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:03.282300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fce24afd-412e-4716-8b09-3c529f6e72e3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:03.283719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fce24afd-412e-4716-8b09-3c529f6e72e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:03.332915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba9e14-e328-496c-a5ae-3b8d9660bae0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:03.338842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09addf98-e665-432f-b831-6a9fd47feb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:03.339503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09addf98-e665-432f-b831-6a9fd47feb8f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:03.339915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09addf98-e665-432f-b831-6a9fd47feb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:03.340235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09addf98-e665-432f-b831-6a9fd47feb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:03.340944 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09addf98-e665-432f-b831-6a9fd47feb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:03.342802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09addf98-e665-432f-b831-6a9fd47feb8f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:03.356759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1034/2070] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:43:03.361441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09addf98-e665-432f-b831-6a9fd47feb8f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:03.365951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1037/2071] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:03.372503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09addf98-e665-432f-b831-6a9fd47feb8f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:03.374624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce24afd-412e-4716-8b09-3c529f6e72e3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:03.377285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fce24afd-412e-4716-8b09-3c529f6e72e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:03.378286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2356a-2f2b-434f-b34b-a688885f500c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:03.378916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2356a-2f2b-434f-b34b-a688885f500c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:03.379278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2356a-2f2b-434f-b34b-a688885f500c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:03.379556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2356a-2f2b-434f-b34b-a688885f500c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:03.380323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2356a-2f2b-434f-b34b-a688885f500c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:03.381995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2356a-2f2b-434f-b34b-a688885f500c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:03.393484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1035/2072] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:03.398702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2356a-2f2b-434f-b34b-a688885f500c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:03.410246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2356a-2f2b-434f-b34b-a688885f500c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:03.415961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74efe14-b1f2-4681-af0e-adba9bfe8e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:03.416689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74efe14-b1f2-4681-af0e-adba9bfe8e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:03.417095 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74efe14-b1f2-4681-af0e-adba9bfe8e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:03.417377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74efe14-b1f2-4681-af0e-adba9bfe8e8d admin admin] PATH_INFO: `/v3/projects/833426205d2f4711b700deb1d90938d7/users/b222d363b0274a8f9932c28db09deec6/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:03.434085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74efe14-b1f2-4681-af0e-adba9bfe8e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=833426205d2f4711b700deb1d90938d7, user_id=b222d363b0274a8f9932c28db09deec6, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:03.438049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74efe14-b1f2-4681-af0e-adba9bfe8e8d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:03.443067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce24afd-412e-4716-8b09-3c529f6e72e3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:03.446649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45cfa7-2ca3-432c-97e5-e21baf03e705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:03.447265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45cfa7-2ca3-432c-97e5-e21baf03e705 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:03.447568 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45cfa7-2ca3-432c-97e5-e21baf03e705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:03.447826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45cfa7-2ca3-432c-97e5-e21baf03e705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:03.457597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45cfa7-2ca3-432c-97e5-e21baf03e705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:03.459036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45cfa7-2ca3-432c-97e5-e21baf03e705 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:03.468739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1038/2073] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:03.549328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1036/2074] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:03 2022] PUT /identity/v3/projects/833426205d2f4711b700deb1d90938d7/users/b222d363b0274a8f9932c28db09deec6/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:43:03.561937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70143b-d6cd-40cb-97b0-6668ec666377 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:03.562225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70143b-d6cd-40cb-97b0-6668ec666377 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:03.562512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70143b-d6cd-40cb-97b0-6668ec666377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:03.593822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fd70143b-d6cd-40cb-97b0-6668ec666377 None None] MFA Rules not processed for user `b222d363b0274a8f9932c28db09deec6`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:03.595055 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fd70143b-d6cd-40cb-97b0-6668ec666377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:03.662654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1039/2075] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:03.667429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74efe14-b1f2-4681-af0e-adba9bfe8e8d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:03.746715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74efe14-b1f2-4681-af0e-adba9bfe8e8d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:03.752385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e53060-887e-4f0c-92cb-91feece90e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:03.752989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e53060-887e-4f0c-92cb-91feece90e13 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:03.753272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e53060-887e-4f0c-92cb-91feece90e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:03.753548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e53060-887e-4f0c-92cb-91feece90e13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:03.754226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e53060-887e-4f0c-92cb-91feece90e13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:03.756298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e53060-887e-4f0c-92cb-91feece90e13 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:03.827535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70143b-d6cd-40cb-97b0-6668ec666377 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:03.829750 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fd70143b-d6cd-40cb-97b0-6668ec666377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:03.841704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:03.846866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70143b-d6cd-40cb-97b0-6668ec666377 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:03.848066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1037/2076] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:03 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:03.852382 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e53060-887e-4f0c-92cb-91feece90e13 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:03.932488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:03.933151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:03.933438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:03.933720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] PATH_INFO: `/v3/projects/26d217c7aed04662ba6f8f1d52fea915` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:03.936745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d217c7aed04662ba6f8f1d52fea915)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:03.939179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:03.971472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e53060-887e-4f0c-92cb-91feece90e13 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:03.976342 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6423236-bfc9-4fdf-85c9-66857236b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:03.976967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6423236-bfc9-4fdf-85c9-66857236b5ec admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:03.977252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6423236-bfc9-4fdf-85c9-66857236b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:03.977529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6423236-bfc9-4fdf-85c9-66857236b5ec admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:03.978197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6423236-bfc9-4fdf-85c9-66857236b5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:03.980154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6423236-bfc9-4fdf-85c9-66857236b5ec admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:04.010793 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:04.078817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1038/2077] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:03 2022] POST /identity/v3/users => generated 295 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:04.083291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6423236-bfc9-4fdf-85c9-66857236b5ec admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:04.089993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d217c7aed04662ba6f8f1d52fea915', 'request_id': 'req-ab71c3e1-cb6d-4795-bca2-56cf070f694b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:04.098542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d217c7aed04662ba6f8f1d52fea915', 'request_id': 'req-ab71c3e1-cb6d-4795-bca2-56cf070f694b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:04.117188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6423236-bfc9-4fdf-85c9-66857236b5ec admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:04.178850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d217c7aed04662ba6f8f1d52fea915', 'request_id': 'req-ab71c3e1-cb6d-4795-bca2-56cf070f694b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:04.188837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9f4cf-7c01-4235-ada8-c16133d3b4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:04.189499 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f4cf-7c01-4235-ada8-c16133d3b4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.189782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f4cf-7c01-4235-ada8-c16133d3b4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.190060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f4cf-7c01-4235-ada8-c16133d3b4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.190781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9f4cf-7c01-4235-ada8-c16133d3b4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:04.192618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9f4cf-7c01-4235-ada8-c16133d3b4f4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:04.195051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] The token cache is being invalidate because project 26d217c7aed04662ba6f8f1d52fea915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:04.195349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:04.200505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:04.205798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1040/2078] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:03 2022] DELETE /identity/v3/projects/26d217c7aed04662ba6f8f1d52fea915 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:04.205798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:04.208751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:04.214900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1039/2079] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:04.217293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665d28d-722e-4313-9645-a38399d66cf0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.217950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665d28d-722e-4313-9645-a38399d66cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.218395 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665d28d-722e-4313-9645-a38399d66cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.230815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71c3e1-cb6d-4795-bca2-56cf070f694b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:04.236166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad97845-f994-4c3e-a37b-48df8bb69dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:04.236811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad97845-f994-4c3e-a37b-48df8bb69dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.237096 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad97845-f994-4c3e-a37b-48df8bb69dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.237368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad97845-f994-4c3e-a37b-48df8bb69dfb admin admin] PATH_INFO: `/v3/users/65e1af5780284e60915b839e172eb7b5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.238561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad97845-f994-4c3e-a37b-48df8bb69dfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=65e1af5780284e60915b839e172eb7b5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:04.240518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad97845-f994-4c3e-a37b-48df8bb69dfb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:04.244852 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c665d28d-722e-4313-9645-a38399d66cf0 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:04.246056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c665d28d-722e-4313-9645-a38399d66cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:04.254787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1040/2080] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:04.255628 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665d28d-722e-4313-9645-a38399d66cf0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:04.257075 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c665d28d-722e-4313-9645-a38399d66cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:04.268753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665d28d-722e-4313-9645-a38399d66cf0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:04.273182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dea27-83f9-465b-856d-2c4142ecc6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:04.273834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dea27-83f9-465b-856d-2c4142ecc6ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.274112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dea27-83f9-465b-856d-2c4142ecc6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.274387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dea27-83f9-465b-856d-2c4142ecc6ff admin admin] PATH_INFO: `/v3/projects/0c9bb82dec524c0b84e8f65f78db291f/users/12d3dbb2c59f49fc9bbeafa28450c70d/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.288755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dea27-83f9-465b-856d-2c4142ecc6ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9bb82dec524c0b84e8f65f78db291f, user_id=12d3dbb2c59f49fc9bbeafa28450c70d, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:04.291442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dea27-83f9-465b-856d-2c4142ecc6ff admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:04.381993 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1041/2081] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:04 2022] PUT /identity/v3/projects/0c9bb82dec524c0b84e8f65f78db291f/users/12d3dbb2c59f49fc9bbeafa28450c70d/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:04.383080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dea27-83f9-465b-856d-2c4142ecc6ff admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:04.384745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-610dea27-83f9-465b-856d-2c4142ecc6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:04.413958 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8ad97845-f994-4c3e-a37b-48df8bb69dfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65e1af5780284e60915b839e172eb7b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:04.414820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8ad97845-f994-4c3e-a37b-48df8bb69dfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65e1af5780284e60915b839e172eb7b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:04.415468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8ad97845-f994-4c3e-a37b-48df8bb69dfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65e1af5780284e60915b839e172eb7b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:04.428676 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8ad97845-f994-4c3e-a37b-48df8bb69dfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65e1af5780284e60915b839e172eb7b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:04.447073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dea27-83f9-465b-856d-2c4142ecc6ff admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:04.450685 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbac8a-c963-4743-af78-fb86d0dfd262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:04.451363 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbac8a-c963-4743-af78-fb86d0dfd262 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.451648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbac8a-c963-4743-af78-fb86d0dfd262 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.451928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbac8a-c963-4743-af78-fb86d0dfd262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.453967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aafbac8a-c963-4743-af78-fb86d0dfd262 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:04.484922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8ad97845-f994-4c3e-a37b-48df8bb69dfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65e1af5780284e60915b839e172eb7b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:04.497461 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1041/2082] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:04 2022] DELETE /identity/v3/users/65e1af5780284e60915b839e172eb7b5 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:04.501442 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e8e96-b13d-4950-82f2-0ed725e53d97 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.501442 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e8e96-b13d-4950-82f2-0ed725e53d97 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.501442 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e8e96-b13d-4950-82f2-0ed725e53d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.514349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbac8a-c963-4743-af78-fb86d0dfd262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:04.514759 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:04.515840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbac8a-c963-4743-af78-fb86d0dfd262 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:04.521249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1d2e8e96-b13d-4950-82f2-0ed725e53d97 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:04.522519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1d2e8e96-b13d-4950-82f2-0ed725e53d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:04.525180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1042/2083] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:04.525985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbac8a-c963-4743-af78-fb86d0dfd262 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:04.527245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aafbac8a-c963-4743-af78-fb86d0dfd262 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:04.595843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1042/2084] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:04.597618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d0024-9808-45db-8003-d140a4f4f989 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.598005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d0024-9808-45db-8003-d140a4f4f989 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.598399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d0024-9808-45db-8003-d140a4f4f989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.606128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbac8a-c963-4743-af78-fb86d0dfd262 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:04.616830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96611b77-bd85-4c2d-b6fc-9478be03128b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:04.617652 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96611b77-bd85-4c2d-b6fc-9478be03128b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.617948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96611b77-bd85-4c2d-b6fc-9478be03128b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.618398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96611b77-bd85-4c2d-b6fc-9478be03128b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.619115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96611b77-bd85-4c2d-b6fc-9478be03128b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:04.621877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96611b77-bd85-4c2d-b6fc-9478be03128b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:04.625658 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bd3d0024-9808-45db-8003-d140a4f4f989 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:04.627129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bd3d0024-9808-45db-8003-d140a4f4f989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:04.636540 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1043/2085] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:04.640185 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1043/2086] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:04.640272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96611b77-bd85-4c2d-b6fc-9478be03128b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:04.648980 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d0024-9808-45db-8003-d140a4f4f989 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:04.652083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96611b77-bd85-4c2d-b6fc-9478be03128b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:04.657147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4c822-ed61-418d-aed5-541cb2a927a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:04.657614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4c822-ed61-418d-aed5-541cb2a927a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.657901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4c822-ed61-418d-aed5-541cb2a927a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.658182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4c822-ed61-418d-aed5-541cb2a927a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.658989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4c822-ed61-418d-aed5-541cb2a927a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:43:04.659236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4c822-ed61-418d-aed5-541cb2a927a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:04.660799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d0024-9808-45db-8003-d140a4f4f989 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:04.660996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4c822-ed61-418d-aed5-541cb2a927a1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:04.666114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bef0b-e5a3-4bd8-afc0-2b3c820dc5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:04.666999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bef0b-e5a3-4bd8-afc0-2b3c820dc5b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.667290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bef0b-e5a3-4bd8-afc0-2b3c820dc5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.667568 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bef0b-e5a3-4bd8-afc0-2b3c820dc5b1 admin admin] PATH_INFO: `/v3/projects/0c9bb82dec524c0b84e8f65f78db291f/users/12d3dbb2c59f49fc9bbeafa28450c70d/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.669682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0bef0b-e5a3-4bd8-afc0-2b3c820dc5b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9bb82dec524c0b84e8f65f78db291f, user_id=12d3dbb2c59f49fc9bbeafa28450c70d, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:04.675027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0bef0b-e5a3-4bd8-afc0-2b3c820dc5b1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:04.676837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1044/2087] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:43:04.677679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4c822-ed61-418d-aed5-541cb2a927a1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:04.679433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d8f4c822-ed61-418d-aed5-541cb2a927a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:04.691589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1044/2088] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:04 2022] PUT /identity/v3/projects/0c9bb82dec524c0b84e8f65f78db291f/users/12d3dbb2c59f49fc9bbeafa28450c70d/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:04.692638 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bef0b-e5a3-4bd8-afc0-2b3c820dc5b1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:04.741212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4c822-ed61-418d-aed5-541cb2a927a1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:04.774003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bef0b-e5a3-4bd8-afc0-2b3c820dc5b1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:04.778774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b8ed3-8da0-42ba-a304-12886a6daabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:04.784211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b8ed3-8da0-42ba-a304-12886a6daabe admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.784211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b8ed3-8da0-42ba-a304-12886a6daabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.784211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b8ed3-8da0-42ba-a304-12886a6daabe admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.784211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b8ed3-8da0-42ba-a304-12886a6daabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:04.784211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b8ed3-8da0-42ba-a304-12886a6daabe admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:04.792218 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2abcb-6ca6-4296-87d9-31aa7fa9d4e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:04.792837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2abcb-6ca6-4296-87d9-31aa7fa9d4e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.793124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2abcb-6ca6-4296-87d9-31aa7fa9d4e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.793403 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2abcb-6ca6-4296-87d9-31aa7fa9d4e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.804154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2abcb-6ca6-4296-87d9-31aa7fa9d4e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:04.805611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2abcb-6ca6-4296-87d9-31aa7fa9d4e0 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:04.816086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1045/2089] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:04.817301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14f7b-46b2-4e6d-96e4-5593fbe22e64 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:04.817590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14f7b-46b2-4e6d-96e4-5593fbe22e64 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:04.817863 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14f7b-46b2-4e6d-96e4-5593fbe22e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:04.846511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-85a14f7b-46b2-4e6d-96e4-5593fbe22e64 None None] MFA Rules not processed for user `12d3dbb2c59f49fc9bbeafa28450c70d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:04.847716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-85a14f7b-46b2-4e6d-96e4-5593fbe22e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:04.903976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1045/2090] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:04 2022] POST /identity/v3/projects => generated 383 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:04.909041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b8ed3-8da0-42ba-a304-12886a6daabe admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:04.965929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1046/2091] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:04.979775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a14f7b-46b2-4e6d-96e4-5593fbe22e64 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:04.983161 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-85a14f7b-46b2-4e6d-96e4-5593fbe22e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:05.020645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b8ed3-8da0-42ba-a304-12886a6daabe admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:05.027211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140c147-cb1f-4476-9dd1-a70c06574c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:05.027907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140c147-cb1f-4476-9dd1-a70c06574c1d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:05.028952 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140c147-cb1f-4476-9dd1-a70c06574c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:05.029230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140c147-cb1f-4476-9dd1-a70c06574c1d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:05.030576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140c147-cb1f-4476-9dd1-a70c06574c1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:05.032366 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140c147-cb1f-4476-9dd1-a70c06574c1d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:05.065732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a14f7b-46b2-4e6d-96e4-5593fbe22e64 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:05.135464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1046/2092] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:04 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:43:05.137140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140c147-cb1f-4476-9dd1-a70c06574c1d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:05.146663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc297e-3109-4c4f-b8ea-8f7de685127a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:05.147311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc297e-3109-4c4f-b8ea-8f7de685127a service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:05.147599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc297e-3109-4c4f-b8ea-8f7de685127a service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:05.147881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc297e-3109-4c4f-b8ea-8f7de685127a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:05.151109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140c147-cb1f-4476-9dd1-a70c06574c1d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:05.157743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9432c6d-edce-419e-a8f5-86e4dcd1e86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:05.158571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9432c6d-edce-419e-a8f5-86e4dcd1e86d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:05.164580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9432c6d-edce-419e-a8f5-86e4dcd1e86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:05.164580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9432c6d-edce-419e-a8f5-86e4dcd1e86d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:05.164580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9432c6d-edce-419e-a8f5-86e4dcd1e86d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:05.164985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9432c6d-edce-419e-a8f5-86e4dcd1e86d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:05.184585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1047/2093] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:43:05.189329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9432c6d-edce-419e-a8f5-86e4dcd1e86d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:05.202565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9432c6d-edce-419e-a8f5-86e4dcd1e86d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:05.210694 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4948ef6-8c6c-4bac-8fe0-8ef8c7c9ed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:05.212380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4948ef6-8c6c-4bac-8fe0-8ef8c7c9ed8a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:05.213188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4948ef6-8c6c-4bac-8fe0-8ef8c7c9ed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:05.213945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4948ef6-8c6c-4bac-8fe0-8ef8c7c9ed8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:05.215039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4948ef6-8c6c-4bac-8fe0-8ef8c7c9ed8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:05.217359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4948ef6-8c6c-4bac-8fe0-8ef8c7c9ed8a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:05.245008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1048/2094] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:43:05.248934 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4948ef6-8c6c-4bac-8fe0-8ef8c7c9ed8a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:05.254552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc297e-3109-4c4f-b8ea-8f7de685127a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:05.259105 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc297e-3109-4c4f-b8ea-8f7de685127a service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:05.264928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4948ef6-8c6c-4bac-8fe0-8ef8c7c9ed8a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:05.272644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3664b19-ded0-4e25-afeb-21ff3b146334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:05.273138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3664b19-ded0-4e25-afeb-21ff3b146334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:05.273426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3664b19-ded0-4e25-afeb-21ff3b146334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:05.273712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3664b19-ded0-4e25-afeb-21ff3b146334 admin admin] PATH_INFO: `/v3/projects/c38a9752cb264f1486f1aa69f4015689/users/16109b0a884d493f969d88ad47671016/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:05.276686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1047/2095] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:05.296090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3664b19-ded0-4e25-afeb-21ff3b146334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38a9752cb264f1486f1aa69f4015689, user_id=16109b0a884d493f969d88ad47671016, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:05.299078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3664b19-ded0-4e25-afeb-21ff3b146334 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:05.343956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:05.403237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1049/2096] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:05 2022] PUT /identity/v3/projects/c38a9752cb264f1486f1aa69f4015689/users/16109b0a884d493f969d88ad47671016/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:05.418093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded732cf-51a2-46cc-b05c-ce831acfe0c5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:05.418382 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded732cf-51a2-46cc-b05c-ce831acfe0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:05.418660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded732cf-51a2-46cc-b05c-ce831acfe0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:05.441151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3664b19-ded0-4e25-afeb-21ff3b146334 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:05.451964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d3664b19-ded0-4e25-afeb-21ff3b146334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:05.500209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ded732cf-51a2-46cc-b05c-ce831acfe0c5 None None] MFA Rules not processed for user `16109b0a884d493f969d88ad47671016`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:05.502838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ded732cf-51a2-46cc-b05c-ce831acfe0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:05.540165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3664b19-ded0-4e25-afeb-21ff3b146334 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:05.556759 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:05.559717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:05.560339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:05.561057 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] PATH_INFO: `/v3/projects/d921271c706843de875563f45467c9c8` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:05.563440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d921271c706843de875563f45467c9c8)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:05.567694 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:05.609225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1048/2097] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:05.610096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded732cf-51a2-46cc-b05c-ce831acfe0c5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:05.627028 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ded732cf-51a2-46cc-b05c-ce831acfe0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:05.688215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded732cf-51a2-46cc-b05c-ce831acfe0c5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:05.691744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382efff-75dc-40bc-9de1-c1c986fd0253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:05.692391 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382efff-75dc-40bc-9de1-c1c986fd0253 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:05.692671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382efff-75dc-40bc-9de1-c1c986fd0253 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:05.692943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382efff-75dc-40bc-9de1-c1c986fd0253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:05.694673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7382efff-75dc-40bc-9de1-c1c986fd0253 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:05.746963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd921271c706843de875563f45467c9c8', 'request_id': 'req-fdf9aac2-7b76-44eb-8fef-891c7f922a63'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:05.756866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd921271c706843de875563f45467c9c8', 'request_id': 'req-fdf9aac2-7b76-44eb-8fef-891c7f922a63'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:05.778199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382efff-75dc-40bc-9de1-c1c986fd0253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:05.780449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382efff-75dc-40bc-9de1-c1c986fd0253 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:05.790941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1049/2098] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:05.791733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382efff-75dc-40bc-9de1-c1c986fd0253 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:05.804567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382efff-75dc-40bc-9de1-c1c986fd0253 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:05.884517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd921271c706843de875563f45467c9c8', 'request_id': 'req-fdf9aac2-7b76-44eb-8fef-891c7f922a63'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:05.897334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe7db9-8b7c-4f98-81da-05cef90f5541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:05.899036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] The token cache is being invalidate because project d921271c706843de875563f45467c9c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:05.899333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:05.900121 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe7db9-8b7c-4f98-81da-05cef90f5541 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:05.900555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe7db9-8b7c-4f98-81da-05cef90f5541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:05.900831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe7db9-8b7c-4f98-81da-05cef90f5541 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:05.901520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe7db9-8b7c-4f98-81da-05cef90f5541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:05.903472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe7db9-8b7c-4f98-81da-05cef90f5541 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:05.904949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fdf9aac2-7b76-44eb-8fef-891c7f922a63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:05.913473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1050/2099] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:05 2022] DELETE /identity/v3/projects/d921271c706843de875563f45467c9c8 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:05.915167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d2e0e-72db-49cb-a932-db60ad81e610 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:05.916207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d2e0e-72db-49cb-a932-db60ad81e610 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:05.916585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d2e0e-72db-49cb-a932-db60ad81e610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:05.942362 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-698d2e0e-72db-49cb-a932-db60ad81e610 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:05.943866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-698d2e0e-72db-49cb-a932-db60ad81e610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:05.981258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1051/2100] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:05.989781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd71e0-1836-4c0b-8cf6-255f9e6fcbe8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:05.990395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd71e0-1836-4c0b-8cf6-255f9e6fcbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:05.990885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd71e0-1836-4c0b-8cf6-255f9e6fcbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.036922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dccd71e0-1836-4c0b-8cf6-255f9e6fcbe8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:06.038835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dccd71e0-1836-4c0b-8cf6-255f9e6fcbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:06.057459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1052/2101] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:06.058941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd71e0-1836-4c0b-8cf6-255f9e6fcbe8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.062876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1050/2102] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:05 2022] POST /identity/v3/projects => generated 381 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:06.063876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe7db9-8b7c-4f98-81da-05cef90f5541 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.065853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-61fe7db9-8b7c-4f98-81da-05cef90f5541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:06.075002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd71e0-1836-4c0b-8cf6-255f9e6fcbe8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:06.180945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:06.184301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe7db9-8b7c-4f98-81da-05cef90f5541 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:06.192306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed00f8-0b07-4961-a604-5dc808d60a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:06.193248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed00f8-0b07-4961-a604-5dc808d60a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.193639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed00f8-0b07-4961-a604-5dc808d60a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.194154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed00f8-0b07-4961-a604-5dc808d60a5c admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.195079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed00f8-0b07-4961-a604-5dc808d60a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:06.197991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed00f8-0b07-4961-a604-5dc808d60a5c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:06.223288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6f816-9dbf-4f34-864e-0f4b27ec29c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:06.224233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6f816-9dbf-4f34-864e-0f4b27ec29c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.224585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6f816-9dbf-4f34-864e-0f4b27ec29c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.224997 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6f816-9dbf-4f34-864e-0f4b27ec29c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.225893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6f816-9dbf-4f34-864e-0f4b27ec29c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:43:06.226404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6f816-9dbf-4f34-864e-0f4b27ec29c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:06.228127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6f816-9dbf-4f34-864e-0f4b27ec29c9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:06.242671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1053/2103] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:43:06.252258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6f816-9dbf-4f34-864e-0f4b27ec29c9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.271464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6f816-9dbf-4f34-864e-0f4b27ec29c9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:06.282695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec354c60-78e4-44ab-8a69-75c04f1ab4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:06.283496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec354c60-78e4-44ab-8a69-75c04f1ab4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.284192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec354c60-78e4-44ab-8a69-75c04f1ab4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.284594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec354c60-78e4-44ab-8a69-75c04f1ab4e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.285398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec354c60-78e4-44ab-8a69-75c04f1ab4e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:06.287505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec354c60-78e4-44ab-8a69-75c04f1ab4e0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:06.291043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1051/2104] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:06 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:43:06.296098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed00f8-0b07-4961-a604-5dc808d60a5c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.307916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed00f8-0b07-4961-a604-5dc808d60a5c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:06.321444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:06.327874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544acc-24fa-4b80-bc8c-0c770cd668ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:06.328719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544acc-24fa-4b80-bc8c-0c770cd668ac admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.329109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544acc-24fa-4b80-bc8c-0c770cd668ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.329497 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544acc-24fa-4b80-bc8c-0c770cd668ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.330766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8544acc-24fa-4b80-bc8c-0c770cd668ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:06.333007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8544acc-24fa-4b80-bc8c-0c770cd668ac admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:06.348897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1052/2105] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:43:06.353377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544acc-24fa-4b80-bc8c-0c770cd668ac admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.367001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544acc-24fa-4b80-bc8c-0c770cd668ac admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:06.374276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e6286-58d7-463e-9368-71d96cc701c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:06.379261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e6286-58d7-463e-9368-71d96cc701c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.379677 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e6286-58d7-463e-9368-71d96cc701c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.380057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e6286-58d7-463e-9368-71d96cc701c8 admin admin] PATH_INFO: `/v3/projects/2d67180f83ba4502943c6bf702b4dceb/users/9e330bbab0c3434f8156a555ba376ed6/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.410645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1054/2106] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:06 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:43:06.414688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec354c60-78e4-44ab-8a69-75c04f1ab4e0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.429128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e6286-58d7-463e-9368-71d96cc701c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d67180f83ba4502943c6bf702b4dceb, user_id=9e330bbab0c3434f8156a555ba376ed6, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:06.434294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e6286-58d7-463e-9368-71d96cc701c8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:06.512673 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec354c60-78e4-44ab-8a69-75c04f1ab4e0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:06.522940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f0c0d-1aa3-4b52-9614-a81a3db2cad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:06.522940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f0c0d-1aa3-4b52-9614-a81a3db2cad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.522940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f0c0d-1aa3-4b52-9614-a81a3db2cad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.522940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f0c0d-1aa3-4b52-9614-a81a3db2cad4 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.523457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f0c0d-1aa3-4b52-9614-a81a3db2cad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:06.523457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f0c0d-1aa3-4b52-9614-a81a3db2cad4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:06.582962 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1053/2107] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:06 2022] PUT /identity/v3/projects/2d67180f83ba4502943c6bf702b4dceb/users/9e330bbab0c3434f8156a555ba376ed6/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:06.585047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e6286-58d7-463e-9368-71d96cc701c8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.615537 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1055/2108] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:06 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:06.624746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f0c0d-1aa3-4b52-9614-a81a3db2cad4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.643189 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f0c0d-1aa3-4b52-9614-a81a3db2cad4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:06.667951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e6286-58d7-463e-9368-71d96cc701c8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:06.673142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7b6f2-e476-4bfa-b620-4c767c629792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:06.673759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b6f2-e476-4bfa-b620-4c767c629792 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.674040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b6f2-e476-4bfa-b620-4c767c629792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.674312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b6f2-e476-4bfa-b620-4c767c629792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.675000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7b6f2-e476-4bfa-b620-4c767c629792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:06.676663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7b6f2-e476-4bfa-b620-4c767c629792 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:06.690455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1054/2109] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:06.691502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7b6f2-e476-4bfa-b620-4c767c629792 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.703003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7b6f2-e476-4bfa-b620-4c767c629792 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:06.705399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184fbe1-ed2e-466b-9ccf-ab837037955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:06.706024 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184fbe1-ed2e-466b-9ccf-ab837037955c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.706580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184fbe1-ed2e-466b-9ccf-ab837037955c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.706872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184fbe1-ed2e-466b-9ccf-ab837037955c admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.707582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184fbe1-ed2e-466b-9ccf-ab837037955c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:06.709287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184fbe1-ed2e-466b-9ccf-ab837037955c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:06.710051 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5c233-014f-45f7-a846-74f81ea3bd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:06.710729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5c233-014f-45f7-a846-74f81ea3bd0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.711038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5c233-014f-45f7-a846-74f81ea3bd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.711321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5c233-014f-45f7-a846-74f81ea3bd0c admin admin] PATH_INFO: `/v3/projects/2d67180f83ba4502943c6bf702b4dceb/users/9e330bbab0c3434f8156a555ba376ed6/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.713522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5c233-014f-45f7-a846-74f81ea3bd0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d67180f83ba4502943c6bf702b4dceb, user_id=9e330bbab0c3434f8156a555ba376ed6, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:06.716200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5c233-014f-45f7-a846-74f81ea3bd0c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:06.719939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1056/2110] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:06.724213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184fbe1-ed2e-466b-9ccf-ab837037955c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.731243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1055/2111] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:06 2022] PUT /identity/v3/projects/2d67180f83ba4502943c6bf702b4dceb/users/9e330bbab0c3434f8156a555ba376ed6/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:06.737027 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184fbe1-ed2e-466b-9ccf-ab837037955c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:06.742104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d38be-357f-4645-81f9-a7c1f178a95e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.742393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d38be-357f-4645-81f9-a7c1f178a95e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.742670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d38be-357f-4645-81f9-a7c1f178a95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.776321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-231d38be-357f-4645-81f9-a7c1f178a95e None None] MFA Rules not processed for user `9e330bbab0c3434f8156a555ba376ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:06.777881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-231d38be-357f-4645-81f9-a7c1f178a95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:06.846953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb254f-45f1-4e41-b8f8-e26a6a60dddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:06.846953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb254f-45f1-4e41-b8f8-e26a6a60dddd admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.846953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb254f-45f1-4e41-b8f8-e26a6a60dddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.847598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb254f-45f1-4e41-b8f8-e26a6a60dddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.852953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb254f-45f1-4e41-b8f8-e26a6a60dddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:06.856225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb254f-45f1-4e41-b8f8-e26a6a60dddd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:06.870681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1057/2112] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:43:06.876329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb254f-45f1-4e41-b8f8-e26a6a60dddd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.908279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb254f-45f1-4e41-b8f8-e26a6a60dddd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:06.916414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc50c1-f170-4378-8b80-dac0cdc36809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:06.917172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc50c1-f170-4378-8b80-dac0cdc36809 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:06.917457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc50c1-f170-4378-8b80-dac0cdc36809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:06.917736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc50c1-f170-4378-8b80-dac0cdc36809 admin admin] PATH_INFO: `/v3/projects/d75287403e8443abbc8d73c2dc7ee50a/users/aeefc5cd1c654490a5406fa5c702fd5a/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:06.938711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc50c1-f170-4378-8b80-dac0cdc36809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75287403e8443abbc8d73c2dc7ee50a, user_id=aeefc5cd1c654490a5406fa5c702fd5a, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:06.941157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc50c1-f170-4378-8b80-dac0cdc36809 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:06.979055 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1056/2113] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:06.996267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d38be-357f-4645-81f9-a7c1f178a95e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:06.996267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-231d38be-357f-4645-81f9-a7c1f178a95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:07.023748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1058/2114] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:06 2022] PUT /identity/v3/projects/d75287403e8443abbc8d73c2dc7ee50a/users/aeefc5cd1c654490a5406fa5c702fd5a/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:07.036978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099acdab-39e2-4c54-a7a4-71975e44da41 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:07.038432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099acdab-39e2-4c54-a7a4-71975e44da41 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:07.038874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099acdab-39e2-4c54-a7a4-71975e44da41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:07.050741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:07.077882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d38be-357f-4645-81f9-a7c1f178a95e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:07.080195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-099acdab-39e2-4c54-a7a4-71975e44da41 None None] MFA Rules not processed for user `aeefc5cd1c654490a5406fa5c702fd5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:07.081685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-099acdab-39e2-4c54-a7a4-71975e44da41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:07.143057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e645e-3635-4a0e-8732-5f3a49a612fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:07.143827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e645e-3635-4a0e-8732-5f3a49a612fd service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:07.144207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e645e-3635-4a0e-8732-5f3a49a612fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:07.144561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e645e-3635-4a0e-8732-5f3a49a612fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:07.185396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1059/2115] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:07.196605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099acdab-39e2-4c54-a7a4-71975e44da41 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:07.199555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-099acdab-39e2-4c54-a7a4-71975e44da41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:07.219311 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099acdab-39e2-4c54-a7a4-71975e44da41 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:07.224372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3076ed7-cd36-4c97-a0fe-af91081dd715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:07.225007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3076ed7-cd36-4c97-a0fe-af91081dd715 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:07.225300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3076ed7-cd36-4c97-a0fe-af91081dd715 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:07.225552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3076ed7-cd36-4c97-a0fe-af91081dd715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:07.237458 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3076ed7-cd36-4c97-a0fe-af91081dd715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:07.238975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3076ed7-cd36-4c97-a0fe-af91081dd715 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:07.249797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1060/2116] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:07.262000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e645e-3635-4a0e-8732-5f3a49a612fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:07.262000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e645e-3635-4a0e-8732-5f3a49a612fd service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:07.273572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1057/2117] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:07.452426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3076ed7-cd36-4c97-a0fe-af91081dd715 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:07.459852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:07.465502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3076ed7-cd36-4c97-a0fe-af91081dd715 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:07.470637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46fc4c-584d-4f3e-a544-f6c25e35d2f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:07.471718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46fc4c-584d-4f3e-a544-f6c25e35d2f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:07.472349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46fc4c-584d-4f3e-a544-f6c25e35d2f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:07.472817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46fc4c-584d-4f3e-a544-f6c25e35d2f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:07.474929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9b46fc4c-584d-4f3e-a544-f6c25e35d2f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:07.566360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46fc4c-584d-4f3e-a544-f6c25e35d2f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:07.567964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46fc4c-584d-4f3e-a544-f6c25e35d2f5 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:07.579543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1061/2118] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:07.771277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:08.384159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e645e-3635-4a0e-8732-5f3a49a612fd service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:08.471364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e645e-3635-4a0e-8732-5f3a49a612fd service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:08.477206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:08.478010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:08.478213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:08.478434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] PATH_INFO: `/v3/users/16109b0a884d493f969d88ad47671016` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:08.479622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=16109b0a884d493f969d88ad47671016)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:08.481711 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:08.507242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:08.773050 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16109b0a884d493f969d88ad47671016'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:08.773694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16109b0a884d493f969d88ad47671016'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:08.781289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16109b0a884d493f969d88ad47671016'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:08.793349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16109b0a884d493f969d88ad47671016'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:08.955105 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16109b0a884d493f969d88ad47671016'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:08.967303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1058/2119] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:08 2022] DELETE /identity/v3/users/16109b0a884d493f969d88ad47671016 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:08.971495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f36cb5-4883-4581-b841-41b6036c9410 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:08.971968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f36cb5-4883-4581-b841-41b6036c9410 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:08.972205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f36cb5-4883-4581-b841-41b6036c9410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:09.009709 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-12f36cb5-4883-4581-b841-41b6036c9410 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:09.011064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-12f36cb5-4883-4581-b841-41b6036c9410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:09.099304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1062/2120] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:09.115170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:09.121048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:09.250884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd569b5d-61ad-4ad6-b3e5-07ee909139cb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:09.257568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40b96e-a20d-457d-ab06-9254b82e7ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:09.258226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b96e-a20d-457d-ab06-9254b82e7ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:09.258532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b96e-a20d-457d-ab06-9254b82e7ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:09.258808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b96e-a20d-457d-ab06-9254b82e7ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:09.274813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40b96e-a20d-457d-ab06-9254b82e7ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:09.276337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40b96e-a20d-457d-ab06-9254b82e7ff3 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:09.292856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1059/2121] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:09.830240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f36cb5-4883-4581-b841-41b6036c9410 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:09.832136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-12f36cb5-4883-4581-b841-41b6036c9410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:09.852268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f36cb5-4883-4581-b841-41b6036c9410 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:09.866683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:09.866683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:09.866683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:09.866683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] PATH_INFO: `/v3/projects/c38a9752cb264f1486f1aa69f4015689` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:09.868472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c38a9752cb264f1486f1aa69f4015689)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:09.870469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:10.069005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c38a9752cb264f1486f1aa69f4015689', 'request_id': 'req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:10.087428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c38a9752cb264f1486f1aa69f4015689', 'request_id': 'req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:10.170748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c38a9752cb264f1486f1aa69f4015689', 'request_id': 'req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:10.192953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] The token cache is being invalidate because project c38a9752cb264f1486f1aa69f4015689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:10.193536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:10.194880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:10.198410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1063/2122] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:09 2022] DELETE /identity/v3/projects/c38a9752cb264f1486f1aa69f4015689 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:10.202912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40b96e-a20d-457d-ab06-9254b82e7ff3 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:10.203377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:10.219187 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40b96e-a20d-457d-ab06-9254b82e7ff3 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:10.353449 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:10.354987 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:10.359033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:10.399635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:10.399635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:10.399635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:10.399635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] PATH_INFO: `/v3/users/9e330bbab0c3434f8156a555ba376ed6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:10.402369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e330bbab0c3434f8156a555ba376ed6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:10.409128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:10.504641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71d9bb-3a8e-4c5e-b9c2-50f4e5de93b9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:10.518144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a34173-555e-4848-a9e2-09e5c06bd6d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:10.519065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a34173-555e-4848-a9e2-09e5c06bd6d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:10.519493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a34173-555e-4848-a9e2-09e5c06bd6d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:10.519946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a34173-555e-4848-a9e2-09e5c06bd6d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:10.522684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-80a34173-555e-4848-a9e2-09e5c06bd6d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:10.658101 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a34173-555e-4848-a9e2-09e5c06bd6d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:10.659607 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a34173-555e-4848-a9e2-09e5c06bd6d1 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:10.663511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e330bbab0c3434f8156a555ba376ed6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:10.664493 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e330bbab0c3434f8156a555ba376ed6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:10.670364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1064/2123] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:10.671636 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424d22e-48aa-45cd-94a3-f5926438aee0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:10.671907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424d22e-48aa-45cd-94a3-f5926438aee0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:10.672217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424d22e-48aa-45cd-94a3-f5926438aee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:10.674125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e330bbab0c3434f8156a555ba376ed6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:10.691126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e330bbab0c3434f8156a555ba376ed6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:10.717177 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b424d22e-48aa-45cd-94a3-f5926438aee0 None None] MFA Rules not processed for user `e5fe46987d5942b3a0ec5495aa0040ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:10.718571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b424d22e-48aa-45cd-94a3-f5926438aee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:10.780544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e330bbab0c3434f8156a555ba376ed6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:10.795591 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1060/2124] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:10 2022] DELETE /identity/v3/users/9e330bbab0c3434f8156a555ba376ed6 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:10.830994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1065/2125] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:10.845971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:10.847424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:10.933451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5861550-fa53-4900-956a-b7fa7081bdc1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:10.940734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1208f-8de3-47d4-81d2-7badce53a87e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:10.941405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1208f-8de3-47d4-81d2-7badce53a87e service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:10.941687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1208f-8de3-47d4-81d2-7badce53a87e service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:10.941961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1208f-8de3-47d4-81d2-7badce53a87e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:11.007941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1208f-8de3-47d4-81d2-7badce53a87e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:11.009866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1208f-8de3-47d4-81d2-7badce53a87e service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:11.019200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1061/2126] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:11.303509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424d22e-48aa-45cd-94a3-f5926438aee0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:11.305024 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b424d22e-48aa-45cd-94a3-f5926438aee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:11.311769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:11.319052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424d22e-48aa-45cd-94a3-f5926438aee0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:11.380787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1208f-8de3-47d4-81d2-7badce53a87e service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:11.382672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7ce1208f-8de3-47d4-81d2-7badce53a87e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:11.385932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af8289-3114-4fa8-a22b-0342fd0ef25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:11.386596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af8289-3114-4fa8-a22b-0342fd0ef25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:11.387463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af8289-3114-4fa8-a22b-0342fd0ef25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:11.387463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af8289-3114-4fa8-a22b-0342fd0ef25c admin admin] PATH_INFO: `/v3/users/c710058b96cb424dbb5b94d2c5d0f278` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:11.388495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af8289-3114-4fa8-a22b-0342fd0ef25c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c710058b96cb424dbb5b94d2c5d0f278)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:11.390637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af8289-3114-4fa8-a22b-0342fd0ef25c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:11.406894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1208f-8de3-47d4-81d2-7badce53a87e service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:11.413092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:11.413776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:11.414131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:11.414511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] PATH_INFO: `/v3/projects/2d67180f83ba4502943c6bf702b4dceb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:11.416029 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d67180f83ba4502943c6bf702b4dceb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:11.418358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:11.718407 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d67180f83ba4502943c6bf702b4dceb', 'request_id': 'req-52315076-786c-405e-810f-0f6207c7c3b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:11.742348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c4af8289-3114-4fa8-a22b-0342fd0ef25c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c710058b96cb424dbb5b94d2c5d0f278'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:11.743429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c4af8289-3114-4fa8-a22b-0342fd0ef25c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c710058b96cb424dbb5b94d2c5d0f278'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:11.743801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c4af8289-3114-4fa8-a22b-0342fd0ef25c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c710058b96cb424dbb5b94d2c5d0f278'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:11.758169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c4af8289-3114-4fa8-a22b-0342fd0ef25c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c710058b96cb424dbb5b94d2c5d0f278'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:11.791089 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d67180f83ba4502943c6bf702b4dceb', 'request_id': 'req-52315076-786c-405e-810f-0f6207c7c3b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:11.797480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d67180f83ba4502943c6bf702b4dceb', 'request_id': 'req-52315076-786c-405e-810f-0f6207c7c3b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:11.808360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] The token cache is being invalidate because project 2d67180f83ba4502943c6bf702b4dceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:11.808669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:11.809312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-52315076-786c-405e-810f-0f6207c7c3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:11.810889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1062/2127] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:11 2022] DELETE /identity/v3/projects/2d67180f83ba4502943c6bf702b4dceb => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:11.815611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65055df-2115-43ed-8205-17789fbd11a3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:11.815892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65055df-2115-43ed-8205-17789fbd11a3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:11.816197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65055df-2115-43ed-8205-17789fbd11a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:11.836462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e65055df-2115-43ed-8205-17789fbd11a3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:11.837741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e65055df-2115-43ed-8205-17789fbd11a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:11.871091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c4af8289-3114-4fa8-a22b-0342fd0ef25c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c710058b96cb424dbb5b94d2c5d0f278'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:11.890103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1066/2128] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:11 2022] DELETE /identity/v3/users/c710058b96cb424dbb5b94d2c5d0f278 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:11.891700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbc01f-9d7f-45a0-8bf6-19155278b1c3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:11.892014 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbc01f-9d7f-45a0-8bf6-19155278b1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:11.892327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbc01f-9d7f-45a0-8bf6-19155278b1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:11.927113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-abfbc01f-9d7f-45a0-8bf6-19155278b1c3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:11.934212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-abfbc01f-9d7f-45a0-8bf6-19155278b1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:11.934474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1063/2129] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:11.937167 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316cfd-4d6e-4daa-a427-7f0886909bed None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:11.937462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316cfd-4d6e-4daa-a427-7f0886909bed None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:11.938358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316cfd-4d6e-4daa-a427-7f0886909bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:11.978398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-45316cfd-4d6e-4daa-a427-7f0886909bed None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:11.979505 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-45316cfd-4d6e-4daa-a427-7f0886909bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:12.020399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1067/2130] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:12.038032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbc01f-9d7f-45a0-8bf6-19155278b1c3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:12.040622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-abfbc01f-9d7f-45a0-8bf6-19155278b1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:12.089942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1064/2131] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:12.095831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45316cfd-4d6e-4daa-a427-7f0886909bed None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:12.108136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45316cfd-4d6e-4daa-a427-7f0886909bed None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:12.113599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2620e-e136-4f2e-a79e-37846d99ea69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:12.117268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2620e-e136-4f2e-a79e-37846d99ea69 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:12.117268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2620e-e136-4f2e-a79e-37846d99ea69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:12.117268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2620e-e136-4f2e-a79e-37846d99ea69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:12.117268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2620e-e136-4f2e-a79e-37846d99ea69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:43:12.117268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2620e-e136-4f2e-a79e-37846d99ea69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:12.119199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2620e-e136-4f2e-a79e-37846d99ea69 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:12.121550 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbc01f-9d7f-45a0-8bf6-19155278b1c3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:12.127671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f890732-702a-49fa-88ea-d7fd5df3295c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:12.128766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f890732-702a-49fa-88ea-d7fd5df3295c service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:12.128766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f890732-702a-49fa-88ea-d7fd5df3295c service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:12.129030 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f890732-702a-49fa-88ea-d7fd5df3295c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:12.136755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1065/2132] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:43:12.137898 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2620e-e136-4f2e-a79e-37846d99ea69 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:12.143730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f890732-702a-49fa-88ea-d7fd5df3295c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:12.145729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f890732-702a-49fa-88ea-d7fd5df3295c service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:12.151039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2620e-e136-4f2e-a79e-37846d99ea69 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:12.159006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58887447-7857-493b-90ca-3b6f699f1c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:12.159006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58887447-7857-493b-90ca-3b6f699f1c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:12.169464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1068/2133] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:12.169691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58887447-7857-493b-90ca-3b6f699f1c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:12.169999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58887447-7857-493b-90ca-3b6f699f1c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:12.174461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58887447-7857-493b-90ca-3b6f699f1c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:12.176553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58887447-7857-493b-90ca-3b6f699f1c54 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:12.194953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f890732-702a-49fa-88ea-d7fd5df3295c service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:12.196953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8f890732-702a-49fa-88ea-d7fd5df3295c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:12.223582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f890732-702a-49fa-88ea-d7fd5df3295c service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:12.227974 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:12.235046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947fb06-e704-4914-b55f-81706a9d9a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:12.235390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947fb06-e704-4914-b55f-81706a9d9a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:12.235722 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947fb06-e704-4914-b55f-81706a9d9a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:12.236009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947fb06-e704-4914-b55f-81706a9d9a12 admin admin] PATH_INFO: `/v3/users/aeefc5cd1c654490a5406fa5c702fd5a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:12.240854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947fb06-e704-4914-b55f-81706a9d9a12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeefc5cd1c654490a5406fa5c702fd5a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:12.242980 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947fb06-e704-4914-b55f-81706a9d9a12 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:12.299207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1066/2134] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:12 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:12.301063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58887447-7857-493b-90ca-3b6f699f1c54 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:12.396320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58887447-7857-493b-90ca-3b6f699f1c54 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:12.402093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad15432-1cc5-4ebd-b808-29ed5cace176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:12.402738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15432-1cc5-4ebd-b808-29ed5cace176 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:12.403025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15432-1cc5-4ebd-b808-29ed5cace176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:12.403313 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15432-1cc5-4ebd-b808-29ed5cace176 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:12.404392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad15432-1cc5-4ebd-b808-29ed5cace176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:12.406359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad15432-1cc5-4ebd-b808-29ed5cace176 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:12.478490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2947fb06-e704-4914-b55f-81706a9d9a12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeefc5cd1c654490a5406fa5c702fd5a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:12.481052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2947fb06-e704-4914-b55f-81706a9d9a12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeefc5cd1c654490a5406fa5c702fd5a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:12.482457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2947fb06-e704-4914-b55f-81706a9d9a12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeefc5cd1c654490a5406fa5c702fd5a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:12.499024 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2947fb06-e704-4914-b55f-81706a9d9a12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeefc5cd1c654490a5406fa5c702fd5a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:12.533210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1067/2135] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:12 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:12.536123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad15432-1cc5-4ebd-b808-29ed5cace176 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:12.537880 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9ad15432-1cc5-4ebd-b808-29ed5cace176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:12.551260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad15432-1cc5-4ebd-b808-29ed5cace176 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:12.556240 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead12de-f36e-44f8-a124-b1879c988ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:12.556881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead12de-f36e-44f8-a124-b1879c988ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:12.557164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead12de-f36e-44f8-a124-b1879c988ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:12.561933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead12de-f36e-44f8-a124-b1879c988ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:12.561933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead12de-f36e-44f8-a124-b1879c988ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:12.561933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead12de-f36e-44f8-a124-b1879c988ae3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:12.577539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1068/2136] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:43:12.582177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead12de-f36e-44f8-a124-b1879c988ae3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:12.604358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead12de-f36e-44f8-a124-b1879c988ae3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:12.610447 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2947fb06-e704-4914-b55f-81706a9d9a12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeefc5cd1c654490a5406fa5c702fd5a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:12.612088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e1362-4469-49f7-99d8-85ae3a977a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:12.612899 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1362-4469-49f7-99d8-85ae3a977a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:12.613326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1362-4469-49f7-99d8-85ae3a977a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:12.613839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1362-4469-49f7-99d8-85ae3a977a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:12.614787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e1362-4469-49f7-99d8-85ae3a977a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:12.619326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e1362-4469-49f7-99d8-85ae3a977a69 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:12.630900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1069/2137] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:12.633614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1069/2138] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:12 2022] DELETE /identity/v3/users/aeefc5cd1c654490a5406fa5c702fd5a => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:12.637131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e1362-4469-49f7-99d8-85ae3a977a69 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:12.637705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe94613-a6dd-4bc8-9ad6-fa6022e23254 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:12.638605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe94613-a6dd-4bc8-9ad6-fa6022e23254 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:12.639057 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe94613-a6dd-4bc8-9ad6-fa6022e23254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:12.667223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-abe94613-a6dd-4bc8-9ad6-fa6022e23254 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:12.668904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-abe94613-a6dd-4bc8-9ad6-fa6022e23254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:12.671208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e1362-4469-49f7-99d8-85ae3a977a69 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:12.785077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c792149-ba33-4ff4-a4e2-3909fde4eb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:12.788109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c792149-ba33-4ff4-a4e2-3909fde4eb82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:12.789048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c792149-ba33-4ff4-a4e2-3909fde4eb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:12.789512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c792149-ba33-4ff4-a4e2-3909fde4eb82 admin admin] PATH_INFO: `/v3/projects/dd79063e46554daba7d75c1cdbae494e/users/c616652bc40141c3ba39877223849437/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:12.809414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c792149-ba33-4ff4-a4e2-3909fde4eb82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd79063e46554daba7d75c1cdbae494e, user_id=c616652bc40141c3ba39877223849437, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:12.813266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c792149-ba33-4ff4-a4e2-3909fde4eb82 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:12.815062 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1070/2139] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:12.826243 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe94613-a6dd-4bc8-9ad6-fa6022e23254 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:12.827745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-abe94613-a6dd-4bc8-9ad6-fa6022e23254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:12.911844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe94613-a6dd-4bc8-9ad6-fa6022e23254 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:12.919729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b1bcf-60f8-45d7-a2d6-468d7e2bb263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:12.920743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b1bcf-60f8-45d7-a2d6-468d7e2bb263 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:12.921530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b1bcf-60f8-45d7-a2d6-468d7e2bb263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:12.922136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b1bcf-60f8-45d7-a2d6-468d7e2bb263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:12.938183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b1bcf-60f8-45d7-a2d6-468d7e2bb263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:12.940887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b1bcf-60f8-45d7-a2d6-468d7e2bb263 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:12.964949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1071/2140] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:12.965853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b1bcf-60f8-45d7-a2d6-468d7e2bb263 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:13.000077 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2a0b1bcf-60f8-45d7-a2d6-468d7e2bb263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:13.001372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b1bcf-60f8-45d7-a2d6-468d7e2bb263 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:13.008547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:13.009316 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:13.009602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:13.009877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] PATH_INFO: `/v3/projects/08c8e191d1bb4d1ea39cbf1c2eead475` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:13.011043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c8e191d1bb4d1ea39cbf1c2eead475)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:13.014352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:13.030334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1070/2141] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:12 2022] PUT /identity/v3/projects/dd79063e46554daba7d75c1cdbae494e/users/c616652bc40141c3ba39877223849437/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:13.047328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d83c2-fa7b-4062-b086-0034447ed225 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:13.047703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d83c2-fa7b-4062-b086-0034447ed225 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:13.048137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d83c2-fa7b-4062-b086-0034447ed225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:13.084098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bb5d83c2-fa7b-4062-b086-0034447ed225 None None] MFA Rules not processed for user `c616652bc40141c3ba39877223849437`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:13.090088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bb5d83c2-fa7b-4062-b086-0034447ed225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:13.164361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1071/2142] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:13.169553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d83c2-fa7b-4062-b086-0034447ed225 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:13.243788 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d83c2-fa7b-4062-b086-0034447ed225 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:13.249084 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6b779-43bd-45c2-90bf-2a3b2e9d8989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:13.249702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6b779-43bd-45c2-90bf-2a3b2e9d8989 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:13.249980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6b779-43bd-45c2-90bf-2a3b2e9d8989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:13.250322 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6b779-43bd-45c2-90bf-2a3b2e9d8989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:13.250916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6b779-43bd-45c2-90bf-2a3b2e9d8989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:13.252853 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6b779-43bd-45c2-90bf-2a3b2e9d8989 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:13.299767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c8e191d1bb4d1ea39cbf1c2eead475', 'request_id': 'req-c6f3d863-cb10-4ed3-89bc-9415e212f951'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:13.306215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c8e191d1bb4d1ea39cbf1c2eead475', 'request_id': 'req-c6f3d863-cb10-4ed3-89bc-9415e212f951'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:13.362965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1072/2143] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:13 2022] POST /identity/v3/projects => generated 383 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:13.366919 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6b779-43bd-45c2-90bf-2a3b2e9d8989 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:13.417616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c8e191d1bb4d1ea39cbf1c2eead475', 'request_id': 'req-c6f3d863-cb10-4ed3-89bc-9415e212f951'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:13.445795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] The token cache is being invalidate because project 08c8e191d1bb4d1ea39cbf1c2eead475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:13.446580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:13.448709 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:13.452998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1072/2144] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:12 2022] DELETE /identity/v3/projects/08c8e191d1bb4d1ea39cbf1c2eead475 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:13.456962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:13.467054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:13.467232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6b779-43bd-45c2-90bf-2a3b2e9d8989 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:13.468460 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a1939-33eb-44b7-836c-858518352748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:13.470081 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a1939-33eb-44b7-836c-858518352748 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:13.470366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a1939-33eb-44b7-836c-858518352748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:13.471096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a1939-33eb-44b7-836c-858518352748 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:13.472547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a1939-33eb-44b7-836c-858518352748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:13.475656 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a1939-33eb-44b7-836c-858518352748 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:13.540238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3d863-cb10-4ed3-89bc-9415e212f951 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:13.546655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:13.551266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:13.551266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:13.551266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] PATH_INFO: `/v3/users/b222d363b0274a8f9932c28db09deec6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:13.552315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b222d363b0274a8f9932c28db09deec6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:13.555801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:13.580423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1073/2145] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:13 2022] POST /identity/v3/users => generated 298 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:43:13.581757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a1939-33eb-44b7-836c-858518352748 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:13.584612 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-541a1939-33eb-44b7-836c-858518352748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:13.599319 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a1939-33eb-44b7-836c-858518352748 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:13.604635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2333a4-b5d5-4d0f-a7d2-c1b826450d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:13.605485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2333a4-b5d5-4d0f-a7d2-c1b826450d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:13.605833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2333a4-b5d5-4d0f-a7d2-c1b826450d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:13.606117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2333a4-b5d5-4d0f-a7d2-c1b826450d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:13.608008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2333a4-b5d5-4d0f-a7d2-c1b826450d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:13.609950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2333a4-b5d5-4d0f-a7d2-c1b826450d84 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:13.620934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1074/2146] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:13.624981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2333a4-b5d5-4d0f-a7d2-c1b826450d84 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:13.641763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2333a4-b5d5-4d0f-a7d2-c1b826450d84 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:13.656250 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2901401-400b-44e8-ab09-c18708a7662c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:13.659443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2901401-400b-44e8-ab09-c18708a7662c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:13.660017 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2901401-400b-44e8-ab09-c18708a7662c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:13.661320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2901401-400b-44e8-ab09-c18708a7662c admin admin] PATH_INFO: `/v3/projects/4e540a3cd0fd41c59890f61537f520c2/users/ae7b8fe0ba6f457ab98d1ed76e556841/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:13.678997 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2901401-400b-44e8-ab09-c18708a7662c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e540a3cd0fd41c59890f61537f520c2, user_id=ae7b8fe0ba6f457ab98d1ed76e556841, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:13.681945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2901401-400b-44e8-ab09-c18708a7662c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:13.812021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1075/2147] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:13 2022] PUT /identity/v3/projects/4e540a3cd0fd41c59890f61537f520c2/users/ae7b8fe0ba6f457ab98d1ed76e556841/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:43:13.814776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2901401-400b-44e8-ab09-c18708a7662c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:13.817506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e2901401-400b-44e8-ab09-c18708a7662c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:13.833716 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b222d363b0274a8f9932c28db09deec6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:13.834430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b222d363b0274a8f9932c28db09deec6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:13.835517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b222d363b0274a8f9932c28db09deec6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:13.854711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b222d363b0274a8f9932c28db09deec6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:13.910412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2901401-400b-44e8-ab09-c18708a7662c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:13.915369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:13.916067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:13.916353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:13.916660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] PATH_INFO: `/v3/projects/d75287403e8443abbc8d73c2dc7ee50a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:13.918125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75287403e8443abbc8d73c2dc7ee50a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:13.920847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:13.947967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b222d363b0274a8f9932c28db09deec6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:13.963957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1073/2148] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:13 2022] DELETE /identity/v3/users/b222d363b0274a8f9932c28db09deec6 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:43:13.964879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:13.969664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:14.086534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75287403e8443abbc8d73c2dc7ee50a', 'request_id': 'req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:14.104301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028c6bc-e624-45ca-a5f2-bc2c266a1082 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:14.194051 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75287403e8443abbc8d73c2dc7ee50a', 'request_id': 'req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:14.206968 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75287403e8443abbc8d73c2dc7ee50a', 'request_id': 'req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:14.208514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9405bd-367a-4281-a5ed-74b6988344b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:14.209275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9405bd-367a-4281-a5ed-74b6988344b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:14.209563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9405bd-367a-4281-a5ed-74b6988344b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:14.209840 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9405bd-367a-4281-a5ed-74b6988344b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:14.210563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9405bd-367a-4281-a5ed-74b6988344b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:14.212963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9405bd-367a-4281-a5ed-74b6988344b4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:14.231205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1074/2149] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:14.235478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] The token cache is being invalidate because project d75287403e8443abbc8d73c2dc7ee50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:14.235776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:14.237165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9405bd-367a-4281-a5ed-74b6988344b4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:14.238164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d7e0729a-a67b-41d6-9720-2f0d2f5993cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:14.241701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bb9405bd-367a-4281-a5ed-74b6988344b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:14.244036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1076/2150] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:13 2022] DELETE /identity/v3/projects/d75287403e8443abbc8d73c2dc7ee50a => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:14.265214 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0b527-a6dd-4c47-a923-12f69a9a4de3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:14.265654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0b527-a6dd-4c47-a923-12f69a9a4de3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:14.266153 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0b527-a6dd-4c47-a923-12f69a9a4de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:14.268654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9405bd-367a-4281-a5ed-74b6988344b4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:14.284533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8a957-358b-4f10-9924-67aab627128f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:14.285340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8a957-358b-4f10-9924-67aab627128f admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:14.285698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8a957-358b-4f10-9924-67aab627128f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:14.286053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8a957-358b-4f10-9924-67aab627128f admin admin] PATH_INFO: `/v3/projects/4e540a3cd0fd41c59890f61537f520c2/users/ae7b8fe0ba6f457ab98d1ed76e556841/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:14.288514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8a957-358b-4f10-9924-67aab627128f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e540a3cd0fd41c59890f61537f520c2, user_id=ae7b8fe0ba6f457ab98d1ed76e556841, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:14.291383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d1c0b527-a6dd-4c47-a923-12f69a9a4de3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:14.292627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1c0b527-a6dd-4c47-a923-12f69a9a4de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:14.293126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8a957-358b-4f10-9924-67aab627128f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:14.306280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1077/2151] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:14.314806 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f42a1-4a8f-48f2-ab5e-58a01206729c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:14.318654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f42a1-4a8f-48f2-ab5e-58a01206729c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:14.318654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f42a1-4a8f-48f2-ab5e-58a01206729c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:14.318886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1075/2152] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:14 2022] PUT /identity/v3/projects/4e540a3cd0fd41c59890f61537f520c2/users/ae7b8fe0ba6f457ab98d1ed76e556841/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:14.332337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac57e46-7976-4f96-a6df-adca9185da92 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:14.332706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac57e46-7976-4f96-a6df-adca9185da92 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:14.333069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac57e46-7976-4f96-a6df-adca9185da92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:14.362657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-741f42a1-4a8f-48f2-ab5e-58a01206729c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:14.365881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-741f42a1-4a8f-48f2-ab5e-58a01206729c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:14.382796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0ac57e46-7976-4f96-a6df-adca9185da92 None None] MFA Rules not processed for user `ae7b8fe0ba6f457ab98d1ed76e556841`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:14.384234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0ac57e46-7976-4f96-a6df-adca9185da92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:14.410724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:14.472605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1078/2153] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:14.473670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f42a1-4a8f-48f2-ab5e-58a01206729c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:14.487134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f42a1-4a8f-48f2-ab5e-58a01206729c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:14.494945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c3cdf-db85-4bf6-9bd9-51e1a73a50b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:14.499568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c3cdf-db85-4bf6-9bd9-51e1a73a50b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:14.499568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c3cdf-db85-4bf6-9bd9-51e1a73a50b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:14.499568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c3cdf-db85-4bf6-9bd9-51e1a73a50b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:14.499568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c3cdf-db85-4bf6-9bd9-51e1a73a50b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:43:14.499568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c3cdf-db85-4bf6-9bd9-51e1a73a50b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:14.500305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c3cdf-db85-4bf6-9bd9-51e1a73a50b9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:14.533929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1079/2154] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:43:14.534145 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1076/2155] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:14.535054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac57e46-7976-4f96-a6df-adca9185da92 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:14.547718 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c3cdf-db85-4bf6-9bd9-51e1a73a50b9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:14.561435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-557c3cdf-db85-4bf6-9bd9-51e1a73a50b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:14.575389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac57e46-7976-4f96-a6df-adca9185da92 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:14.596259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80e292-20c0-4d2c-b89c-58b0226c7520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:14.597294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80e292-20c0-4d2c-b89c-58b0226c7520 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:14.598944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80e292-20c0-4d2c-b89c-58b0226c7520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:14.599228 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80e292-20c0-4d2c-b89c-58b0226c7520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:14.601331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80e292-20c0-4d2c-b89c-58b0226c7520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:14.608926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80e292-20c0-4d2c-b89c-58b0226c7520 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:14.648008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c3cdf-db85-4bf6-9bd9-51e1a73a50b9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:14.652906 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafd8bf-8865-4894-bd4d-17692f60136c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:14.653365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd8bf-8865-4894-bd4d-17692f60136c service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:14.654447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd8bf-8865-4894-bd4d-17692f60136c service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:14.654999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafd8bf-8865-4894-bd4d-17692f60136c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:14.662530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8bafd8bf-8865-4894-bd4d-17692f60136c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:14.685952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:14.725810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafd8bf-8865-4894-bd4d-17692f60136c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:14.727322 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafd8bf-8865-4894-bd4d-17692f60136c service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:14.751401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1080/2156] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:14.752895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae0b9b-8643-4e9d-b93d-a5dd3bac4837 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:14.753189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae0b9b-8643-4e9d-b93d-a5dd3bac4837 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:14.753471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae0b9b-8643-4e9d-b93d-a5dd3bac4837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:14.759436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1077/2157] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:14 2022] POST /identity/v3/projects => generated 363 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:43:14.761279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80e292-20c0-4d2c-b89c-58b0226c7520 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:14.774285 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b5ae0b9b-8643-4e9d-b93d-a5dd3bac4837 None None] MFA Rules not processed for user `e5fe46987d5942b3a0ec5495aa0040ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:14.777101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b5ae0b9b-8643-4e9d-b93d-a5dd3bac4837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:14.867393 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80e292-20c0-4d2c-b89c-58b0226c7520 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:14.874436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18756964-8337-4f91-ae56-405f9af33bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:14.875230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18756964-8337-4f91-ae56-405f9af33bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:14.875593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18756964-8337-4f91-ae56-405f9af33bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:14.875949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18756964-8337-4f91-ae56-405f9af33bd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:14.877062 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18756964-8337-4f91-ae56-405f9af33bd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:14.879321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18756964-8337-4f91-ae56-405f9af33bd4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:14.879821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1081/2158] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:15.008598 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1078/2159] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:14 2022] POST /identity/v3/users => generated 282 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:15.010253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae0b9b-8643-4e9d-b93d-a5dd3bac4837 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:15.028723 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae0b9b-8643-4e9d-b93d-a5dd3bac4837 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:15.033585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04055fec-3d2c-4f1d-a37f-a40508176c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:15.035329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04055fec-3d2c-4f1d-a37f-a40508176c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:15.035329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04055fec-3d2c-4f1d-a37f-a40508176c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:15.035329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04055fec-3d2c-4f1d-a37f-a40508176c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:15.035633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04055fec-3d2c-4f1d-a37f-a40508176c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:15.037247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04055fec-3d2c-4f1d-a37f-a40508176c97 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:15.049613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1082/2160] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:15.053687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18756964-8337-4f91-ae56-405f9af33bd4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:15.065272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18756964-8337-4f91-ae56-405f9af33bd4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:15.070528 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c193a23-5834-4514-9838-a12ef08105a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:15.071424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c193a23-5834-4514-9838-a12ef08105a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:15.071713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c193a23-5834-4514-9838-a12ef08105a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:15.072049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c193a23-5834-4514-9838-a12ef08105a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:15.072745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c193a23-5834-4514-9838-a12ef08105a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:15.074421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c193a23-5834-4514-9838-a12ef08105a8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:15.088104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1079/2161] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:15.092543 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04055fec-3d2c-4f1d-a37f-a40508176c97 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:15.103749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04055fec-3d2c-4f1d-a37f-a40508176c97 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:15.109936 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b19f52-506e-44c0-a389-d0474359a131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:15.110840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b19f52-506e-44c0-a389-d0474359a131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:15.111044 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b19f52-506e-44c0-a389-d0474359a131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:15.111270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b19f52-506e-44c0-a389-d0474359a131 admin admin] PATH_INFO: `/v3/projects/52702e4907c6468a8c107dd533564da6/users/640c3a1c1a3a4fb2a53272d0117e605d/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:15.128753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c193a23-5834-4514-9838-a12ef08105a8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:15.130613 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4c193a23-5834-4514-9838-a12ef08105a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:15.131627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b19f52-506e-44c0-a389-d0474359a131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52702e4907c6468a8c107dd533564da6, user_id=640c3a1c1a3a4fb2a53272d0117e605d, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:15.134804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b19f52-506e-44c0-a389-d0474359a131 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:15.147276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c193a23-5834-4514-9838-a12ef08105a8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:15.152337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:15.152981 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:15.153521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:15.154200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] PATH_INFO: `/v3/projects/833426205d2f4711b700deb1d90938d7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:15.155387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=833426205d2f4711b700deb1d90938d7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:15.157591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:15.230585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1083/2162] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:15 2022] PUT /identity/v3/projects/52702e4907c6468a8c107dd533564da6/users/640c3a1c1a3a4fb2a53272d0117e605d/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:43:15.241977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27bbc5-e28b-452f-95fc-386a585fb85b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:15.242267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27bbc5-e28b-452f-95fc-386a585fb85b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:15.242548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27bbc5-e28b-452f-95fc-386a585fb85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:15.278023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2c27bbc5-e28b-452f-95fc-386a585fb85b None None] MFA Rules not processed for user `640c3a1c1a3a4fb2a53272d0117e605d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:15.280305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2c27bbc5-e28b-452f-95fc-386a585fb85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:15.311642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '833426205d2f4711b700deb1d90938d7', 'request_id': 'req-fd207dd7-8434-42ec-bc4b-5840b217ed07'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:15.320527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '833426205d2f4711b700deb1d90938d7', 'request_id': 'req-fd207dd7-8434-42ec-bc4b-5840b217ed07'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:15.380146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1084/2163] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:15.392367 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27bbc5-e28b-452f-95fc-386a585fb85b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:15.393764 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2c27bbc5-e28b-452f-95fc-386a585fb85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:15.428399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '833426205d2f4711b700deb1d90938d7', 'request_id': 'req-fd207dd7-8434-42ec-bc4b-5840b217ed07'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:15.444457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] The token cache is being invalidate because project 833426205d2f4711b700deb1d90938d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:15.445163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:15.445830 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:15.447372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1080/2164] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:15 2022] DELETE /identity/v3/projects/833426205d2f4711b700deb1d90938d7 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:15.453098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:15.454494 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:15.459458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27bbc5-e28b-452f-95fc-386a585fb85b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:15.517305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259defcb-5f66-422d-b74e-b0e3e6bd5514 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfc4dfda424d40b7c9f17345b8ad45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:15.524487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259defcb-5f66-422d-b74e-b0e3e6bd5514 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:15.528245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259defcb-5f66-422d-b74e-b0e3e6bd5514 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:15.528245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259defcb-5f66-422d-b74e-b0e3e6bd5514 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:15.528245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-259defcb-5f66-422d-b74e-b0e3e6bd5514 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:15.531511 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd207dd7-8434-42ec-bc4b-5840b217ed07 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:15.542878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:15.543839 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:15.544443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:15.544904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] PATH_INFO: `/v3/users/12d3dbb2c59f49fc9bbeafa28450c70d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:15.547456 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d3dbb2c59f49fc9bbeafa28450c70d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:15.550321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:15.623356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259defcb-5f66-422d-b74e-b0e3e6bd5514 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:15.624883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259defcb-5f66-422d-b74e-b0e3e6bd5514 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:15.646659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1085/2165] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:15.790501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d3dbb2c59f49fc9bbeafa28450c70d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:15.791867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d3dbb2c59f49fc9bbeafa28450c70d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:15.792559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d3dbb2c59f49fc9bbeafa28450c70d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:15.806538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d3dbb2c59f49fc9bbeafa28450c70d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:15.930993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d3dbb2c59f49fc9bbeafa28450c70d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:15.952026 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1081/2166] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:15 2022] DELETE /identity/v3/users/12d3dbb2c59f49fc9bbeafa28450c70d => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:16.209602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259defcb-5f66-422d-b74e-b0e3e6bd5514 service glance] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:16.211233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-259defcb-5f66-422d-b74e-b0e3e6bd5514 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:16.225604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259defcb-5f66-422d-b74e-b0e3e6bd5514 service glance] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:16.292756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf852ab4-74c1-4958-8234-f44871ed37ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:16.294518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf852ab4-74c1-4958-8234-f44871ed37ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:16.294518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf852ab4-74c1-4958-8234-f44871ed37ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:16.294518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf852ab4-74c1-4958-8234-f44871ed37ff admin admin] PATH_INFO: `/v3/users/8b6ddc4823424e37b1e13879d970ad62` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:16.295735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf852ab4-74c1-4958-8234-f44871ed37ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b6ddc4823424e37b1e13879d970ad62)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:16.297809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf852ab4-74c1-4958-8234-f44871ed37ff admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:16.425959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:16.426770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:16.454877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338d4f7-ae8d-46a9-a7bb-44781737c5b5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:16.458745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:16.459957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:16.460325 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:16.460600 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] PATH_INFO: `/v3/projects/0c9bb82dec524c0b84e8f65f78db291f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:16.461780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9bb82dec524c0b84e8f65f78db291f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:16.463720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:16.550940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf852ab4-74c1-4958-8234-f44871ed37ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b6ddc4823424e37b1e13879d970ad62'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:16.551705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf852ab4-74c1-4958-8234-f44871ed37ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b6ddc4823424e37b1e13879d970ad62'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:16.558791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf852ab4-74c1-4958-8234-f44871ed37ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b6ddc4823424e37b1e13879d970ad62'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:16.572029 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf852ab4-74c1-4958-8234-f44871ed37ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b6ddc4823424e37b1e13879d970ad62'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:16.605711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9bb82dec524c0b84e8f65f78db291f', 'request_id': 'req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:16.613121 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9bb82dec524c0b84e8f65f78db291f', 'request_id': 'req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:16.678085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf852ab4-74c1-4958-8234-f44871ed37ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b6ddc4823424e37b1e13879d970ad62'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:16.691729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1086/2167] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:16 2022] DELETE /identity/v3/users/8b6ddc4823424e37b1e13879d970ad62 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:16.692540 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817b3ce-2464-41b2-b7f4-86f1e4e7f0fa None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:16.692844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817b3ce-2464-41b2-b7f4-86f1e4e7f0fa None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:16.694266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817b3ce-2464-41b2-b7f4-86f1e4e7f0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:16.719149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d817b3ce-2464-41b2-b7f4-86f1e4e7f0fa None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:16.720675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d817b3ce-2464-41b2-b7f4-86f1e4e7f0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:16.774673 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9bb82dec524c0b84e8f65f78db291f', 'request_id': 'req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:16.783326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1087/2168] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:16.788746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] The token cache is being invalidate because project 0c9bb82dec524c0b84e8f65f78db291f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:16.789119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:16.789913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec064932-8ba5-4bc5-9f2d-1e2a54a63333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:16.791973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1082/2169] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:16 2022] DELETE /identity/v3/projects/0c9bb82dec524c0b84e8f65f78db291f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:43:16.793455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817b3ce-2464-41b2-b7f4-86f1e4e7f0fa None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:16.795950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d817b3ce-2464-41b2-b7f4-86f1e4e7f0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:16.798347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e374c0b-3a41-486d-87e4-b972a300d0b9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:16.798714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e374c0b-3a41-486d-87e4-b972a300d0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:16.799070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e374c0b-3a41-486d-87e4-b972a300d0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:16.821326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0e374c0b-3a41-486d-87e4-b972a300d0b9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:16.822560 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0e374c0b-3a41-486d-87e4-b972a300d0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:16.841700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1083/2170] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:16.850122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fea07-bb22-4a3b-bd5d-3cfed0846a8a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:16.850415 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fea07-bb22-4a3b-bd5d-3cfed0846a8a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:16.850693 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fea07-bb22-4a3b-bd5d-3cfed0846a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:16.860360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817b3ce-2464-41b2-b7f4-86f1e4e7f0fa None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:16.864170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2720be-0225-4800-b320-c70e77249e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:16.864904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2720be-0225-4800-b320-c70e77249e89 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:16.865124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2720be-0225-4800-b320-c70e77249e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:16.865339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2720be-0225-4800-b320-c70e77249e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:16.867129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ce2720be-0225-4800-b320-c70e77249e89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:16.872049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f71fea07-bb22-4a3b-bd5d-3cfed0846a8a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:16.873742 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f71fea07-bb22-4a3b-bd5d-3cfed0846a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:16.880728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2720be-0225-4800-b320-c70e77249e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:16.882139 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2720be-0225-4800-b320-c70e77249e89 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:16.886527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1084/2171] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:16.887565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fea07-bb22-4a3b-bd5d-3cfed0846a8a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:16.894719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1088/2172] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:16.897843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fea07-bb22-4a3b-bd5d-3cfed0846a8a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:16.902929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8244d8-1539-488f-a3b5-fdb0147c167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:16.904429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8244d8-1539-488f-a3b5-fdb0147c167c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:16.904719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8244d8-1539-488f-a3b5-fdb0147c167c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:16.905001 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8244d8-1539-488f-a3b5-fdb0147c167c admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:16.905738 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8244d8-1539-488f-a3b5-fdb0147c167c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:43:16.906046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8244d8-1539-488f-a3b5-fdb0147c167c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:16.907645 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8244d8-1539-488f-a3b5-fdb0147c167c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:16.931411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1085/2173] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:43:16.932693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2720be-0225-4800-b320-c70e77249e89 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:16.943387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2720be-0225-4800-b320-c70e77249e89 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:16.948707 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900027c-4797-4da9-b482-ab1634829604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:16.949460 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900027c-4797-4da9-b482-ab1634829604 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:16.949773 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900027c-4797-4da9-b482-ab1634829604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:16.950794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900027c-4797-4da9-b482-ab1634829604 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:16.951370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900027c-4797-4da9-b482-ab1634829604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:16.953308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900027c-4797-4da9-b482-ab1634829604 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:17.120094 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1089/2174] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:16 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:17.124747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8244d8-1539-488f-a3b5-fdb0147c167c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:17.135586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8244d8-1539-488f-a3b5-fdb0147c167c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:17.146978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:17.226509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f922fe-5e76-4564-a074-9457fff94f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:17.227317 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f922fe-5e76-4564-a074-9457fff94f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:17.227886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f922fe-5e76-4564-a074-9457fff94f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:17.227886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f922fe-5e76-4564-a074-9457fff94f48 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:17.228757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f922fe-5e76-4564-a074-9457fff94f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:17.230688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f922fe-5e76-4564-a074-9457fff94f48 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:17.331894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1086/2175] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:17 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:17.336933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900027c-4797-4da9-b482-ab1634829604 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:17.350328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900027c-4797-4da9-b482-ab1634829604 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:17.355113 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb01b9-6e72-4b0a-a9f4-16ad4c990d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:17.355735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb01b9-6e72-4b0a-a9f4-16ad4c990d88 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:17.356052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb01b9-6e72-4b0a-a9f4-16ad4c990d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:17.356335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb01b9-6e72-4b0a-a9f4-16ad4c990d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:17.357287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb01b9-6e72-4b0a-a9f4-16ad4c990d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:17.358942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb01b9-6e72-4b0a-a9f4-16ad4c990d88 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:17.375372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1090/2176] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:17.380301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f922fe-5e76-4564-a074-9457fff94f48 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:17.392912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f922fe-5e76-4564-a074-9457fff94f48 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:17.397719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681640c-1997-4493-952f-dd5b6ae37d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:17.401217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681640c-1997-4493-952f-dd5b6ae37d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:17.401217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681640c-1997-4493-952f-dd5b6ae37d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:17.401217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681640c-1997-4493-952f-dd5b6ae37d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:17.401217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681640c-1997-4493-952f-dd5b6ae37d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:17.401550 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681640c-1997-4493-952f-dd5b6ae37d59 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:17.412538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1087/2177] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:17.419029 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb01b9-6e72-4b0a-a9f4-16ad4c990d88 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:17.431228 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb01b9-6e72-4b0a-a9f4-16ad4c990d88 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:17.439527 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:17.442344 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681640c-1997-4493-952f-dd5b6ae37d59 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:17.444251 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1681640c-1997-4493-952f-dd5b6ae37d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:17.448219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f36f7-5031-4f12-9fd7-130bd622da03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:17.449223 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f36f7-5031-4f12-9fd7-130bd622da03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:17.449678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f36f7-5031-4f12-9fd7-130bd622da03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:17.450104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f36f7-5031-4f12-9fd7-130bd622da03 admin admin] PATH_INFO: `/v3/projects/5f9c8164f84445c5b93f9ee4762a03b8/users/336d8a07fde947d58102654e9d786102/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:17.464553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681640c-1997-4493-952f-dd5b6ae37d59 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:17.471937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:17.473421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:17.473963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:17.474597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] PATH_INFO: `/v3/projects/6051d2259fb34f0292d82ba737345069` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:17.476419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6051d2259fb34f0292d82ba737345069)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:17.478656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:17.504197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f36f7-5031-4f12-9fd7-130bd622da03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9c8164f84445c5b93f9ee4762a03b8, user_id=336d8a07fde947d58102654e9d786102, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:17.508562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f36f7-5031-4f12-9fd7-130bd622da03 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:17.650769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1091/2178] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:17 2022] PUT /identity/v3/projects/5f9c8164f84445c5b93f9ee4762a03b8/users/336d8a07fde947d58102654e9d786102/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:17.662670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b40ff-9fb6-483c-8e5e-e8446176685e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:17.662959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b40ff-9fb6-483c-8e5e-e8446176685e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:17.663260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b40ff-9fb6-483c-8e5e-e8446176685e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:17.666652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6051d2259fb34f0292d82ba737345069', 'request_id': 'req-46bc5d4b-ff62-4060-9059-d318dfba520c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:17.673599 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6051d2259fb34f0292d82ba737345069', 'request_id': 'req-46bc5d4b-ff62-4060-9059-d318dfba520c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:17.702752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a07b40ff-9fb6-483c-8e5e-e8446176685e None None] MFA Rules not processed for user `336d8a07fde947d58102654e9d786102`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:17.704061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a07b40ff-9fb6-483c-8e5e-e8446176685e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:17.773447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1092/2179] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:17.787380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b40ff-9fb6-483c-8e5e-e8446176685e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:17.788359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6051d2259fb34f0292d82ba737345069', 'request_id': 'req-46bc5d4b-ff62-4060-9059-d318dfba520c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:17.790522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a07b40ff-9fb6-483c-8e5e-e8446176685e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:17.803047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] The token cache is being invalidate because project 6051d2259fb34f0292d82ba737345069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:17.803703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:17.804129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:17.806036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1088/2180] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:17 2022] DELETE /identity/v3/projects/6051d2259fb34f0292d82ba737345069 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:43:17.810219 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:17.815164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:17.863580 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b40ff-9fb6-483c-8e5e-e8446176685e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:17.867419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c1188-6edc-4b3f-abb2-62681b4c6cca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:17.872602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c1188-6edc-4b3f-abb2-62681b4c6cca service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:17.872602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c1188-6edc-4b3f-abb2-62681b4c6cca service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:17.872602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c1188-6edc-4b3f-abb2-62681b4c6cca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:17.872602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-738c1188-6edc-4b3f-abb2-62681b4c6cca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:17.884354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c1188-6edc-4b3f-abb2-62681b4c6cca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:17.885805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c1188-6edc-4b3f-abb2-62681b4c6cca service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:17.898278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1093/2181] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:17.906716 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc5d4b-ff62-4060-9059-d318dfba520c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:17.911229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:17.912139 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:17.912520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:17.912989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] PATH_INFO: `/v3/users/d60ed3a7fdb04887bc801c213ec47440` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:17.914262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] RBAC: Authorizing `identity:delete_user(user_id=d60ed3a7fdb04887bc801c213ec47440)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:17.917056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:18.145996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd60ed3a7fdb04887bc801c213ec47440'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:18.146692 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd60ed3a7fdb04887bc801c213ec47440'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:18.147389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd60ed3a7fdb04887bc801c213ec47440'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:18.160073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd60ed3a7fdb04887bc801c213ec47440'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:18.227204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd60ed3a7fdb04887bc801c213ec47440'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:18.241735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1089/2182] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:17 2022] DELETE /identity/v3/users/d60ed3a7fdb04887bc801c213ec47440 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:18.845309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c1188-6edc-4b3f-abb2-62681b4c6cca service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:18.857112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c1188-6edc-4b3f-abb2-62681b4c6cca service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:18.942320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:18.942961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:18.943312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:18.943587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] PATH_INFO: `/v3/projects/88c8eb581d2c46c8a006effaf641e034` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:18.944768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=88c8eb581d2c46c8a006effaf641e034)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:18.946709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:19.084333 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c8eb581d2c46c8a006effaf641e034', 'request_id': 'req-739838f6-4768-4309-ba97-e82019627eea'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:19.181799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c8eb581d2c46c8a006effaf641e034', 'request_id': 'req-739838f6-4768-4309-ba97-e82019627eea'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:19.188585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c8eb581d2c46c8a006effaf641e034', 'request_id': 'req-739838f6-4768-4309-ba97-e82019627eea'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:19.200744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] The token cache is being invalidate because project 88c8eb581d2c46c8a006effaf641e034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:19.201031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:19.201977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:19.206952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1094/2183] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:18 2022] DELETE /identity/v3/projects/88c8eb581d2c46c8a006effaf641e034 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:19.210229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:19.211780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:19.225738 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec97b8-9d88-4d38-b80e-3097c438e4af admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:19.336922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:19.338098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:19.338589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:19.339072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] PATH_INFO: `/v3/users/16eea5863af3457e92903b620378edeb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:19.340866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] RBAC: Authorizing `identity:delete_user(user_id=16eea5863af3457e92903b620378edeb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:19.343372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:19.565957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16eea5863af3457e92903b620378edeb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:19.566606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16eea5863af3457e92903b620378edeb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:19.567215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16eea5863af3457e92903b620378edeb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:19.576564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:19.581576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16eea5863af3457e92903b620378edeb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:19.710927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16eea5863af3457e92903b620378edeb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:19.726603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1090/2184] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:19 2022] DELETE /identity/v3/users/16eea5863af3457e92903b620378edeb => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:19.821836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:20.243872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:20.245643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:20.274572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739838f6-4768-4309-ba97-e82019627eea admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:20.342807 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:20.343694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:20.344591 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:20.345651 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] PATH_INFO: `/v3/projects/ec2272c1d63f40a09baa16853c6a84dc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:20.347132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2272c1d63f40a09baa16853c6a84dc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:20.349905 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:20.487048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:20.489226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:20.500595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2272c1d63f40a09baa16853c6a84dc', 'request_id': 'req-9b8e8bdf-615d-41ae-aaae-86b78412dde2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:20.553468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaaa147-3733-42da-b581-7568f90d421b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:20.611854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f449c5-5eac-4fcd-9411-f5c1ab1ae20f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:20.612522 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f449c5-5eac-4fcd-9411-f5c1ab1ae20f service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:20.612977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f449c5-5eac-4fcd-9411-f5c1ab1ae20f service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:20.613261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f449c5-5eac-4fcd-9411-f5c1ab1ae20f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:20.615367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f4f449c5-5eac-4fcd-9411-f5c1ab1ae20f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:20.619542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2272c1d63f40a09baa16853c6a84dc', 'request_id': 'req-9b8e8bdf-615d-41ae-aaae-86b78412dde2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:20.626088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2272c1d63f40a09baa16853c6a84dc', 'request_id': 'req-9b8e8bdf-615d-41ae-aaae-86b78412dde2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:20.640744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] The token cache is being invalidate because project ec2272c1d63f40a09baa16853c6a84dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:20.640744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:20.640744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9b8e8bdf-615d-41ae-aaae-86b78412dde2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:20.641593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1095/2185] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:20 2022] DELETE /identity/v3/projects/ec2272c1d63f40a09baa16853c6a84dc => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:20.647770 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e7e8f-27c6-47e9-9410-8d07dbfdce8c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:20.648124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e7e8f-27c6-47e9-9410-8d07dbfdce8c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:20.648406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e7e8f-27c6-47e9-9410-8d07dbfdce8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:20.672841 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-df7e7e8f-27c6-47e9-9410-8d07dbfdce8c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:20.674188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-df7e7e8f-27c6-47e9-9410-8d07dbfdce8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:20.691909 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f449c5-5eac-4fcd-9411-f5c1ab1ae20f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:20.693527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f449c5-5eac-4fcd-9411-f5c1ab1ae20f service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:20.705560 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1091/2186] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:20.754085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1096/2187] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:20.758956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f5b67-9d3c-4e64-a68d-e8982d489013 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:20.759224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f5b67-9d3c-4e64-a68d-e8982d489013 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:20.759514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f5b67-9d3c-4e64-a68d-e8982d489013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:20.805624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-678f5b67-9d3c-4e64-a68d-e8982d489013 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:20.816162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-678f5b67-9d3c-4e64-a68d-e8982d489013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:20.829914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1092/2188] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:20.830566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e7e8f-27c6-47e9-9410-8d07dbfdce8c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:20.843446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e7e8f-27c6-47e9-9410-8d07dbfdce8c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:20.848804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e8ed5-5adb-4da2-864b-7723407562e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:20.849514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e8ed5-5adb-4da2-864b-7723407562e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:20.849877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e8ed5-5adb-4da2-864b-7723407562e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:20.850463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e8ed5-5adb-4da2-864b-7723407562e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:20.851001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e8ed5-5adb-4da2-864b-7723407562e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:43:20.851385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e8ed5-5adb-4da2-864b-7723407562e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:20.853713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e8ed5-5adb-4da2-864b-7723407562e5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:20.870733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1097/2189] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:43:20.875266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f5b67-9d3c-4e64-a68d-e8982d489013 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:20.884772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f5b67-9d3c-4e64-a68d-e8982d489013 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:20.890200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b50b4-cc9d-4486-a4b7-8e03bdae221e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:20.890828 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b50b4-cc9d-4486-a4b7-8e03bdae221e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:20.891112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b50b4-cc9d-4486-a4b7-8e03bdae221e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:20.891747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b50b4-cc9d-4486-a4b7-8e03bdae221e admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:20.892131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b50b4-cc9d-4486-a4b7-8e03bdae221e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:20.894094 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b50b4-cc9d-4486-a4b7-8e03bdae221e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:20.991388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1093/2190] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:20 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:20.992626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e8ed5-5adb-4da2-864b-7723407562e5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:21.017678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e8ed5-5adb-4da2-864b-7723407562e5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:21.086417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab14c6-b42a-4ab6-86e8-2372ee17a715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:21.087211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab14c6-b42a-4ab6-86e8-2372ee17a715 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:21.087495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab14c6-b42a-4ab6-86e8-2372ee17a715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:21.087773 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab14c6-b42a-4ab6-86e8-2372ee17a715 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:21.088496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab14c6-b42a-4ab6-86e8-2372ee17a715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:21.091227 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab14c6-b42a-4ab6-86e8-2372ee17a715 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:21.133829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b50b4-cc9d-4486-a4b7-8e03bdae221e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:21.135610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-653b50b4-cc9d-4486-a4b7-8e03bdae221e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:21.154862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b50b4-cc9d-4486-a4b7-8e03bdae221e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:21.159575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc61a6-1348-4d58-979f-d82caa33ddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:21.160297 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc61a6-1348-4d58-979f-d82caa33ddbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:21.170329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:21.174020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc61a6-1348-4d58-979f-d82caa33ddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:21.174322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc61a6-1348-4d58-979f-d82caa33ddbd admin admin] PATH_INFO: `/v3/users/336d8a07fde947d58102654e9d786102` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:21.176572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc61a6-1348-4d58-979f-d82caa33ddbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=336d8a07fde947d58102654e9d786102)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:21.178976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc61a6-1348-4d58-979f-d82caa33ddbd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:21.197361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1098/2191] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:20 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:43:21.202118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab14c6-b42a-4ab6-86e8-2372ee17a715 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:21.214004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab14c6-b42a-4ab6-86e8-2372ee17a715 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:21.220506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a61821-a6c8-4de7-a69d-3f049e775989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:21.221132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a61821-a6c8-4de7-a69d-3f049e775989 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:21.221421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a61821-a6c8-4de7-a69d-3f049e775989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:21.221701 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a61821-a6c8-4de7-a69d-3f049e775989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:21.222393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a61821-a6c8-4de7-a69d-3f049e775989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:21.224155 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a61821-a6c8-4de7-a69d-3f049e775989 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:21.241974 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1099/2192] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:21.243316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a61821-a6c8-4de7-a69d-3f049e775989 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:21.254149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a61821-a6c8-4de7-a69d-3f049e775989 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:21.259336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346894d1-2099-4429-a345-bccf40def3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:21.260102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346894d1-2099-4429-a345-bccf40def3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:21.260451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346894d1-2099-4429-a345-bccf40def3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:21.260795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346894d1-2099-4429-a345-bccf40def3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:21.261559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346894d1-2099-4429-a345-bccf40def3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:21.263859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346894d1-2099-4429-a345-bccf40def3c1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:21.278844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1100/2193] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:43:21.280193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346894d1-2099-4429-a345-bccf40def3c1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:21.291633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346894d1-2099-4429-a345-bccf40def3c1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:21.297571 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef39cb1-ca6b-41db-b9bb-9fe55cfb2e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:21.298303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39cb1-ca6b-41db-b9bb-9fe55cfb2e99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:21.301827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39cb1-ca6b-41db-b9bb-9fe55cfb2e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:21.301827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39cb1-ca6b-41db-b9bb-9fe55cfb2e99 admin admin] PATH_INFO: `/v3/projects/8d037ddd73f14dda82de565f8530d9ee/users/b882f6093b6743dabd6c964b086842b0/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:21.315252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef39cb1-ca6b-41db-b9bb-9fe55cfb2e99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d037ddd73f14dda82de565f8530d9ee, user_id=b882f6093b6743dabd6c964b086842b0, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:21.319248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef39cb1-ca6b-41db-b9bb-9fe55cfb2e99 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:21.368574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-90bc61a6-1348-4d58-979f-d82caa33ddbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336d8a07fde947d58102654e9d786102'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:21.369275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-90bc61a6-1348-4d58-979f-d82caa33ddbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336d8a07fde947d58102654e9d786102'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:21.369903 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-90bc61a6-1348-4d58-979f-d82caa33ddbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336d8a07fde947d58102654e9d786102'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:21.383297 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-90bc61a6-1348-4d58-979f-d82caa33ddbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336d8a07fde947d58102654e9d786102'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:21.450717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1101/2194] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:21 2022] PUT /identity/v3/projects/8d037ddd73f14dda82de565f8530d9ee/users/b882f6093b6743dabd6c964b086842b0/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:21.455883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d51e2-0b6d-477f-96fe-7595b8880c2c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:21.456460 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d51e2-0b6d-477f-96fe-7595b8880c2c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:21.456744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d51e2-0b6d-477f-96fe-7595b8880c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:21.475202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-90bc61a6-1348-4d58-979f-d82caa33ddbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336d8a07fde947d58102654e9d786102'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:21.482908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d59d51e2-0b6d-477f-96fe-7595b8880c2c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:21.484436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d59d51e2-0b6d-477f-96fe-7595b8880c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:21.491690 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1094/2195] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:21 2022] DELETE /identity/v3/users/336d8a07fde947d58102654e9d786102 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:43:21.495683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa97309-5d0b-4cfa-84e7-0461e8b3205d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:21.496454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa97309-5d0b-4cfa-84e7-0461e8b3205d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:21.496738 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa97309-5d0b-4cfa-84e7-0461e8b3205d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:21.525107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-caa97309-5d0b-4cfa-84e7-0461e8b3205d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:21.526392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-caa97309-5d0b-4cfa-84e7-0461e8b3205d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:21.600262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:21.641840 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1095/2196] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:21.655644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa97309-5d0b-4cfa-84e7-0461e8b3205d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:21.657567 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-caa97309-5d0b-4cfa-84e7-0461e8b3205d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:21.745346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa97309-5d0b-4cfa-84e7-0461e8b3205d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:21.751276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c3644-3fb5-4af0-a8aa-5171094796b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:21.752315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c3644-3fb5-4af0-a8aa-5171094796b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:21.752706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c3644-3fb5-4af0-a8aa-5171094796b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:21.756438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1102/2197] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:21.756616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c3644-3fb5-4af0-a8aa-5171094796b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:21.764795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d51e2-0b6d-477f-96fe-7595b8880c2c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:21.772002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c3644-3fb5-4af0-a8aa-5171094796b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:21.773468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c3644-3fb5-4af0-a8aa-5171094796b6 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:21.784418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1096/2198] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:21.863286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d51e2-0b6d-477f-96fe-7595b8880c2c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:21.867468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b795c68-f81f-43fc-82f8-5a21fde405b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:21.868118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b795c68-f81f-43fc-82f8-5a21fde405b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:21.868396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b795c68-f81f-43fc-82f8-5a21fde405b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:21.868668 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b795c68-f81f-43fc-82f8-5a21fde405b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:21.883232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b795c68-f81f-43fc-82f8-5a21fde405b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:21.885301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b795c68-f81f-43fc-82f8-5a21fde405b0 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:21.905752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1103/2199] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:22.280896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c3644-3fb5-4af0-a8aa-5171094796b6 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:22.282401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-383c3644-3fb5-4af0-a8aa-5171094796b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:22.303707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c3644-3fb5-4af0-a8aa-5171094796b6 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:22.310107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:22.311326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:22.311751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:22.312264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] PATH_INFO: `/v3/projects/5f9c8164f84445c5b93f9ee4762a03b8` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:22.313973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f9c8164f84445c5b93f9ee4762a03b8)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:22.316897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:22.379377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:22.454612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f9c8164f84445c5b93f9ee4762a03b8', 'request_id': 'req-26a87bdf-48e6-42c6-b301-97db739a202d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:22.461294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f9c8164f84445c5b93f9ee4762a03b8', 'request_id': 'req-26a87bdf-48e6-42c6-b301-97db739a202d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:22.545768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f9c8164f84445c5b93f9ee4762a03b8', 'request_id': 'req-26a87bdf-48e6-42c6-b301-97db739a202d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:22.559115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] The token cache is being invalidate because project 5f9c8164f84445c5b93f9ee4762a03b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:22.559414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:22.560688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-26a87bdf-48e6-42c6-b301-97db739a202d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:22.562386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1097/2200] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:22 2022] DELETE /identity/v3/projects/5f9c8164f84445c5b93f9ee4762a03b8 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:43:22.567583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747a35c-4ff7-4c53-8ca6-20ce32cc1b82 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:22.567875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747a35c-4ff7-4c53-8ca6-20ce32cc1b82 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:22.568186 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747a35c-4ff7-4c53-8ca6-20ce32cc1b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:22.589881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f747a35c-4ff7-4c53-8ca6-20ce32cc1b82 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:22.591193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f747a35c-4ff7-4c53-8ca6-20ce32cc1b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:22.655349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1104/2201] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:22.663962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ddbfb-215a-4c76-9cdf-35338009b525 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:22.664281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ddbfb-215a-4c76-9cdf-35338009b525 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:22.664561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ddbfb-215a-4c76-9cdf-35338009b525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:22.689832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3c7ddbfb-215a-4c76-9cdf-35338009b525 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:22.691394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3c7ddbfb-215a-4c76-9cdf-35338009b525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:22.700642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1098/2202] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:22.705653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747a35c-4ff7-4c53-8ca6-20ce32cc1b82 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:22.719897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747a35c-4ff7-4c53-8ca6-20ce32cc1b82 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:22.725940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe373c-a5dc-478b-bdda-ecd794cfd6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:22.726968 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe373c-a5dc-478b-bdda-ecd794cfd6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:22.728241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe373c-a5dc-478b-bdda-ecd794cfd6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:22.728241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe373c-a5dc-478b-bdda-ecd794cfd6e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:22.728241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe373c-a5dc-478b-bdda-ecd794cfd6e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:43:22.728664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe373c-a5dc-478b-bdda-ecd794cfd6e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:22.730647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe373c-a5dc-478b-bdda-ecd794cfd6e9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:22.749047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1105/2203] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:43:22.750812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ddbfb-215a-4c76-9cdf-35338009b525 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:22.766023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ddbfb-215a-4c76-9cdf-35338009b525 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:22.771102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a5551-b1c1-49dc-a4e3-af3ff118f519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:22.771943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a5551-b1c1-49dc-a4e3-af3ff118f519 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:22.772425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a5551-b1c1-49dc-a4e3-af3ff118f519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:22.772870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a5551-b1c1-49dc-a4e3-af3ff118f519 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:22.773709 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a5551-b1c1-49dc-a4e3-af3ff118f519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:22.776033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a5551-b1c1-49dc-a4e3-af3ff118f519 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:22.886165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1099/2204] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:22 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:43:22.890381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe373c-a5dc-478b-bdda-ecd794cfd6e9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:22.900348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe373c-a5dc-478b-bdda-ecd794cfd6e9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:22.964697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61331dae-e2f7-409c-9cac-88d4995de987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:22.964697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61331dae-e2f7-409c-9cac-88d4995de987 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:22.964697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61331dae-e2f7-409c-9cac-88d4995de987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:22.964697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61331dae-e2f7-409c-9cac-88d4995de987 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:22.965866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61331dae-e2f7-409c-9cac-88d4995de987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:22.967639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61331dae-e2f7-409c-9cac-88d4995de987 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:23.033430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1106/2205] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:22 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:23.037789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a5551-b1c1-49dc-a4e3-af3ff118f519 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:23.049238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a5551-b1c1-49dc-a4e3-af3ff118f519 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:23.054298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776bedd-af4e-4825-9d02-7f958e7b13cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:23.054942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776bedd-af4e-4825-9d02-7f958e7b13cb admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:23.055247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776bedd-af4e-4825-9d02-7f958e7b13cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:23.056017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776bedd-af4e-4825-9d02-7f958e7b13cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:23.056278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776bedd-af4e-4825-9d02-7f958e7b13cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:23.057980 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776bedd-af4e-4825-9d02-7f958e7b13cb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:23.069836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1100/2206] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:23.073847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61331dae-e2f7-409c-9cac-88d4995de987 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:23.086666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61331dae-e2f7-409c-9cac-88d4995de987 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:23.092454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c6b80-5ed9-4ff5-bf0b-c6ed616905a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:23.094001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c6b80-5ed9-4ff5-bf0b-c6ed616905a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:23.094547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c6b80-5ed9-4ff5-bf0b-c6ed616905a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:23.094908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c6b80-5ed9-4ff5-bf0b-c6ed616905a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:23.095843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c6b80-5ed9-4ff5-bf0b-c6ed616905a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:23.097889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c6b80-5ed9-4ff5-bf0b-c6ed616905a6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:23.117568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1107/2207] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:23.122138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776bedd-af4e-4825-9d02-7f958e7b13cb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:23.134218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776bedd-af4e-4825-9d02-7f958e7b13cb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:23.142257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182c86f-50f5-43a0-b1af-71868add0bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:23.142950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182c86f-50f5-43a0-b1af-71868add0bca admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:23.143258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182c86f-50f5-43a0-b1af-71868add0bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:23.143519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182c86f-50f5-43a0-b1af-71868add0bca admin admin] PATH_INFO: `/v3/projects/77dfcb84e1ed40c18f5629ed176a2251/users/108869c217b14a9cb6b5888f8bd94433/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:23.160024 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182c86f-50f5-43a0-b1af-71868add0bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=77dfcb84e1ed40c18f5629ed176a2251, user_id=108869c217b14a9cb6b5888f8bd94433, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:23.162372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182c86f-50f5-43a0-b1af-71868add0bca admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:23.248566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1101/2208] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:23 2022] PUT /identity/v3/projects/77dfcb84e1ed40c18f5629ed176a2251/users/108869c217b14a9cb6b5888f8bd94433/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:43:23.323276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21333e91-6558-4e5b-b132-da0b6ad5d38b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:23.323566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21333e91-6558-4e5b-b132-da0b6ad5d38b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:23.323842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21333e91-6558-4e5b-b132-da0b6ad5d38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:23.328906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:23.359264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3af63-b0cb-4832-8650-bb42b0780d61 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:23.359264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3af63-b0cb-4832-8650-bb42b0780d61 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:23.359264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3af63-b0cb-4832-8650-bb42b0780d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:23.385753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-21333e91-6558-4e5b-b132-da0b6ad5d38b None None] MFA Rules not processed for user `108869c217b14a9cb6b5888f8bd94433`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:23.387014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21333e91-6558-4e5b-b132-da0b6ad5d38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:23.428286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-58c3af63-b0cb-4832-8650-bb42b0780d61 None None] MFA Rules not processed for user `b882f6093b6743dabd6c964b086842b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:23.428286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-58c3af63-b0cb-4832-8650-bb42b0780d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:23.471044 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1108/2209] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:23.490932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21333e91-6558-4e5b-b132-da0b6ad5d38b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:23.494979 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21333e91-6558-4e5b-b132-da0b6ad5d38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:23.539739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1102/2210] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:43:23.552235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3af63-b0cb-4832-8650-bb42b0780d61 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:23.553649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-58c3af63-b0cb-4832-8650-bb42b0780d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:23.574896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21333e91-6558-4e5b-b132-da0b6ad5d38b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:23.586738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630586e6-6ac9-427b-a0e1-342373c0f97d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:23.587648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630586e6-6ac9-427b-a0e1-342373c0f97d service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:23.587911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630586e6-6ac9-427b-a0e1-342373c0f97d service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:23.588386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630586e6-6ac9-427b-a0e1-342373c0f97d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:23.599474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630586e6-6ac9-427b-a0e1-342373c0f97d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:23.601072 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630586e6-6ac9-427b-a0e1-342373c0f97d service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:23.610858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1109/2211] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:23.616577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3af63-b0cb-4832-8650-bb42b0780d61 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:23.620603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69641537-a0a2-4ba4-a673-97c4f7f03b50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:23.621249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69641537-a0a2-4ba4-a673-97c4f7f03b50 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:23.621699 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69641537-a0a2-4ba4-a673-97c4f7f03b50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:23.621806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69641537-a0a2-4ba4-a673-97c4f7f03b50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:23.631126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69641537-a0a2-4ba4-a673-97c4f7f03b50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:23.632619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69641537-a0a2-4ba4-a673-97c4f7f03b50 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:23.644274 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1103/2212] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:24.141893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:25.065223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:25.328289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:25.620171 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80615920-5643-4c9e-a882-779dcdcda05f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:25.620489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80615920-5643-4c9e-a882-779dcdcda05f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:25.620766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80615920-5643-4c9e-a882-779dcdcda05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:25.649354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-80615920-5643-4c9e-a882-779dcdcda05f None None] MFA Rules not processed for user `b882f6093b6743dabd6c964b086842b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:25.650630 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-80615920-5643-4c9e-a882-779dcdcda05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:25.658745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1110/2213] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:25.873845 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:26.685745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:27.224104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69641537-a0a2-4ba4-a673-97c4f7f03b50 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:27.225562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-69641537-a0a2-4ba4-a673-97c4f7f03b50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:27.295898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69641537-a0a2-4ba4-a673-97c4f7f03b50 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:27.302422 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:27.303247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:27.303639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:27.303963 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] PATH_INFO: `/v3/users/c616652bc40141c3ba39877223849437` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:27.305411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c616652bc40141c3ba39877223849437)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:27.310879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:27.647388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c616652bc40141c3ba39877223849437'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:27.648308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c616652bc40141c3ba39877223849437'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:27.649642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c616652bc40141c3ba39877223849437'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:27.666329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c616652bc40141c3ba39877223849437'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:27.736772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c616652bc40141c3ba39877223849437'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:27.749909 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1104/2214] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:27 2022] DELETE /identity/v3/users/c616652bc40141c3ba39877223849437 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:27.754724 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc961b1-752b-43f2-9604-064b97130d26 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:27.755298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc961b1-752b-43f2-9604-064b97130d26 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:27.755438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc961b1-752b-43f2-9604-064b97130d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:27.775671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2bc961b1-752b-43f2-9604-064b97130d26 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:27.777066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2bc961b1-752b-43f2-9604-064b97130d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:27.847496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1111/2215] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:27.854739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:27.856276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:27.918424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465c282-7beb-46c7-8d20-975fafd3b041 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:27.923608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5c43e-2617-4ad9-9c21-ae1b10b700f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:27.924280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5c43e-2617-4ad9-9c21-ae1b10b700f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:27.924557 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5c43e-2617-4ad9-9c21-ae1b10b700f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:27.924837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5c43e-2617-4ad9-9c21-ae1b10b700f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:27.934865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5c43e-2617-4ad9-9c21-ae1b10b700f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:27.936353 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5c43e-2617-4ad9-9c21-ae1b10b700f9 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:27.953833 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1105/2216] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:28.217428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:28.554358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc961b1-752b-43f2-9604-064b97130d26 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:28.558086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2bc961b1-752b-43f2-9604-064b97130d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:28.582195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc961b1-752b-43f2-9604-064b97130d26 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:28.592961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:28.593669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:28.594024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:28.594311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] PATH_INFO: `/v3/projects/dd79063e46554daba7d75c1cdbae494e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:28.595709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd79063e46554daba7d75c1cdbae494e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:28.597707 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:28.757449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd79063e46554daba7d75c1cdbae494e', 'request_id': 'req-9e21b5fd-9000-46ae-9d90-8d87cc92d203'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:28.918790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd79063e46554daba7d75c1cdbae494e', 'request_id': 'req-9e21b5fd-9000-46ae-9d90-8d87cc92d203'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:28.926129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd79063e46554daba7d75c1cdbae494e', 'request_id': 'req-9e21b5fd-9000-46ae-9d90-8d87cc92d203'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:28.939144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] The token cache is being invalidate because project dd79063e46554daba7d75c1cdbae494e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:28.939342 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:28.940048 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:28.941541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1112/2217] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:28 2022] DELETE /identity/v3/projects/dd79063e46554daba7d75c1cdbae494e => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:28.945772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5c43e-2617-4ad9-9c21-ae1b10b700f9 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:28.956248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5c43e-2617-4ad9-9c21-ae1b10b700f9 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:29.015713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:29.016437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:29.016700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:29.016981 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] PATH_INFO: `/v3/users/ae7b8fe0ba6f457ab98d1ed76e556841` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:29.018177 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae7b8fe0ba6f457ab98d1ed76e556841)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:29.020196 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:29.258470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae7b8fe0ba6f457ab98d1ed76e556841'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:29.259133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae7b8fe0ba6f457ab98d1ed76e556841'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:29.259764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae7b8fe0ba6f457ab98d1ed76e556841'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:29.275371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae7b8fe0ba6f457ab98d1ed76e556841'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:29.380778 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=12,keystone:SELECT=83,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:29.383141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae7b8fe0ba6f457ab98d1ed76e556841'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:29.397593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1106/2218] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:28 2022] DELETE /identity/v3/users/ae7b8fe0ba6f457ab98d1ed76e556841 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:29.982090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:29.983600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:30.001299 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21b5fd-9000-46ae-9d90-8d87cc92d203 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:30.073396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:30.074040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:30.074320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:30.074594 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] PATH_INFO: `/v3/projects/4e540a3cd0fd41c59890f61537f520c2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:30.075753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e540a3cd0fd41c59890f61537f520c2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:30.077956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:30.153250 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:30.209446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e540a3cd0fd41c59890f61537f520c2', 'request_id': 'req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:30.301645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e540a3cd0fd41c59890f61537f520c2', 'request_id': 'req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:30.308791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e540a3cd0fd41c59890f61537f520c2', 'request_id': 'req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:30.323182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] The token cache is being invalidate because project 4e540a3cd0fd41c59890f61537f520c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:30.323472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:30.324574 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5fcedcf7-13a5-45a9-97e3-89e269fd53f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:30.326388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1113/2219] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:29 2022] DELETE /identity/v3/projects/4e540a3cd0fd41c59890f61537f520c2 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:31.316497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:31.487938 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:32.198765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:32.421481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:32.717559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:33.342285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:33.867456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:35.290261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:36.384876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:36.527250 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:36.528826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:36.543115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bc20c-7f1f-40cf-b12e-1d6f97095726 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:36.607175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:36.607874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:36.608202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:36.608478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] PATH_INFO: `/v3/users/108869c217b14a9cb6b5888f8bd94433` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:36.609910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=108869c217b14a9cb6b5888f8bd94433)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:36.612002 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:36.952950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108869c217b14a9cb6b5888f8bd94433'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:36.954146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108869c217b14a9cb6b5888f8bd94433'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:36.955071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108869c217b14a9cb6b5888f8bd94433'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:36.970042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108869c217b14a9cb6b5888f8bd94433'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:37.107008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108869c217b14a9cb6b5888f8bd94433'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:37.120614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1107/2220] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:36 2022] DELETE /identity/v3/users/108869c217b14a9cb6b5888f8bd94433 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:37.125737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2b44d-fcbf-44e3-b821-59677b3f9cf6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:37.126230 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2b44d-fcbf-44e3-b821-59677b3f9cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:37.126230 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2b44d-fcbf-44e3-b821-59677b3f9cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:37.145063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dad2b44d-fcbf-44e3-b821-59677b3f9cf6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:37.149833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dad2b44d-fcbf-44e3-b821-59677b3f9cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:37.206736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1114/2221] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:37.216632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:37.217928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:37.268285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:37.278059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2503e36-ff50-4e2e-92e6-4f7e3d388b6d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:37.282879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9b19e-01e0-4573-b1e2-c35684b71c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:37.283495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9b19e-01e0-4573-b1e2-c35684b71c6b service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:37.283801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9b19e-01e0-4573-b1e2-c35684b71c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:37.284098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9b19e-01e0-4573-b1e2-c35684b71c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:37.293081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9b19e-01e0-4573-b1e2-c35684b71c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:37.294526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9b19e-01e0-4573-b1e2-c35684b71c6b service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:37.304634 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1108/2222] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:43:37.843142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2b44d-fcbf-44e3-b821-59677b3f9cf6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:37.847539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dad2b44d-fcbf-44e3-b821-59677b3f9cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:37.863090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2b44d-fcbf-44e3-b821-59677b3f9cf6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:37.869043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:37.869962 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:37.870403 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:37.870861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] PATH_INFO: `/v3/projects/77dfcb84e1ed40c18f5629ed176a2251` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:37.872253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=77dfcb84e1ed40c18f5629ed176a2251)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:37.874309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:38.013964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77dfcb84e1ed40c18f5629ed176a2251', 'request_id': 'req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:38.095262 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77dfcb84e1ed40c18f5629ed176a2251', 'request_id': 'req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:38.101935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77dfcb84e1ed40c18f5629ed176a2251', 'request_id': 'req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:38.114484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] The token cache is being invalidate because project 77dfcb84e1ed40c18f5629ed176a2251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:38.114776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:38.115402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-429ec2d3-64ac-4a5d-858f-1b3c788ecc8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:38.117001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1115/2223] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:37 2022] DELETE /identity/v3/projects/77dfcb84e1ed40c18f5629ed176a2251 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:47.516812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=60,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:48.111326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=59,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:53.869213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9b19e-01e0-4573-b1e2-c35684b71c6b service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:53.870433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4ef9b19e-01e0-4573-b1e2-c35684b71c6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:53.883746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9b19e-01e0-4573-b1e2-c35684b71c6b service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:53.943974 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:53.944643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:53.945003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:53.945258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] PATH_INFO: `/v3/users/640c3a1c1a3a4fb2a53272d0117e605d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:53.948103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=640c3a1c1a3a4fb2a53272d0117e605d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:53.948414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:54.160692 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '640c3a1c1a3a4fb2a53272d0117e605d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:54.161295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '640c3a1c1a3a4fb2a53272d0117e605d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:54.161904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '640c3a1c1a3a4fb2a53272d0117e605d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:54.173878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '640c3a1c1a3a4fb2a53272d0117e605d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:54.267838 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '640c3a1c1a3a4fb2a53272d0117e605d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:54.297665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1109/2224] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:43:53 2022] DELETE /identity/v3/users/640c3a1c1a3a4fb2a53272d0117e605d => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:54.302478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be928b4-e75a-4e9e-90ee-da3f83eb54cb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:54.303326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be928b4-e75a-4e9e-90ee-da3f83eb54cb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:54.303751 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be928b4-e75a-4e9e-90ee-da3f83eb54cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:54.341858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5be928b4-e75a-4e9e-90ee-da3f83eb54cb None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:54.343693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5be928b4-e75a-4e9e-90ee-da3f83eb54cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:54.419653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1116/2225] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:54.427278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:54.428747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:54.503064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50215320-23ba-4990-bac2-2c59f7292561 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:54.507323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c44cd1-6ada-4a13-9335-cd45d6dab33a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:54.508054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c44cd1-6ada-4a13-9335-cd45d6dab33a service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:54.508406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c44cd1-6ada-4a13-9335-cd45d6dab33a service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:54.508806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c44cd1-6ada-4a13-9335-cd45d6dab33a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:54.519513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c44cd1-6ada-4a13-9335-cd45d6dab33a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:54.521845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c44cd1-6ada-4a13-9335-cd45d6dab33a service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:54.533067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1110/2226] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:43:54.782479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6711909-8f52-4183-8258-c660c368bb89 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:54.784263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6711909-8f52-4183-8258-c660c368bb89 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:54.784263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6711909-8f52-4183-8258-c660c368bb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:54.802366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d6711909-8f52-4183-8258-c660c368bb89 None None] MFA Rules not processed for user `b882f6093b6743dabd6c964b086842b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:54.803589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d6711909-8f52-4183-8258-c660c368bb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:54.867037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1117/2227] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:55.042623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae0609-d352-45ec-b0f3-32edc8a551e6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:55.042623 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae0609-d352-45ec-b0f3-32edc8a551e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:55.043208 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae0609-d352-45ec-b0f3-32edc8a551e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:55.064734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e6ae0609-d352-45ec-b0f3-32edc8a551e6 None None] MFA Rules not processed for user `5509519254574e50a70c84ae895acd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:55.065960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e6ae0609-d352-45ec-b0f3-32edc8a551e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:55.083161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:55.136068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1111/2228] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:55.187828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6711909-8f52-4183-8258-c660c368bb89 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:55.189414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d6711909-8f52-4183-8258-c660c368bb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:55.208243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6711909-8f52-4183-8258-c660c368bb89 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:55.214058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:55.216203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:55.216203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:55.216203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] PATH_INFO: `/v3/projects/52702e4907c6468a8c107dd533564da6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:55.217271 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=52702e4907c6468a8c107dd533564da6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:55.220224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:55.374058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52702e4907c6468a8c107dd533564da6', 'request_id': 'req-2b467996-af88-421c-bf5f-75ef892aed8d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:55.472232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52702e4907c6468a8c107dd533564da6', 'request_id': 'req-2b467996-af88-421c-bf5f-75ef892aed8d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:55.479183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52702e4907c6468a8c107dd533564da6', 'request_id': 'req-2b467996-af88-421c-bf5f-75ef892aed8d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:55.490922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] The token cache is being invalidate because project 52702e4907c6468a8c107dd533564da6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:43:55.491273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:55.491873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2b467996-af88-421c-bf5f-75ef892aed8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:43:55.493801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1118/2229] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:43:55 2022] DELETE /identity/v3/projects/52702e4907c6468a8c107dd533564da6 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:55.498649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7ea51-1498-41ae-ad0f-97bd13e06c32 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:55.498898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7ea51-1498-41ae-ad0f-97bd13e06c32 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:55.499864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7ea51-1498-41ae-ad0f-97bd13e06c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:55.520470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dcc7ea51-1498-41ae-ad0f-97bd13e06c32 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:55.521683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dcc7ea51-1498-41ae-ad0f-97bd13e06c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:55.583761 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1112/2230] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:55.591405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a18a9b-4fe7-4926-b57f-fd1155758714 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:55.591694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a18a9b-4fe7-4926-b57f-fd1155758714 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:55.591969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a18a9b-4fe7-4926-b57f-fd1155758714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:55.612768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e2a18a9b-4fe7-4926-b57f-fd1155758714 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:55.613975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e2a18a9b-4fe7-4926-b57f-fd1155758714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:55.622871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1119/2231] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:55.627787 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7ea51-1498-41ae-ad0f-97bd13e06c32 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:55.637766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7ea51-1498-41ae-ad0f-97bd13e06c32 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:55.643377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54554ac-85be-4e49-a104-568b132cccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:55.644090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54554ac-85be-4e49-a104-568b132cccf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:55.647434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54554ac-85be-4e49-a104-568b132cccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:55.647434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54554ac-85be-4e49-a104-568b132cccf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:55.647434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54554ac-85be-4e49-a104-568b132cccf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:43:55.647434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54554ac-85be-4e49-a104-568b132cccf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:55.647434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54554ac-85be-4e49-a104-568b132cccf8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:55.660428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1113/2232] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:43:55.665008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a18a9b-4fe7-4926-b57f-fd1155758714 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:55.675565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a18a9b-4fe7-4926-b57f-fd1155758714 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:55.680425 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09367e-f1bb-40a1-ad9d-502b5c3a582d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:55.681118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09367e-f1bb-40a1-ad9d-502b5c3a582d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:55.681645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09367e-f1bb-40a1-ad9d-502b5c3a582d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:55.682030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09367e-f1bb-40a1-ad9d-502b5c3a582d admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:55.682832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09367e-f1bb-40a1-ad9d-502b5c3a582d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:55.684985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09367e-f1bb-40a1-ad9d-502b5c3a582d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:55.768453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:55.774980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1120/2233] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:43:55 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:55.779785 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54554ac-85be-4e49-a104-568b132cccf8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:55.793520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54554ac-85be-4e49-a104-568b132cccf8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:55.857446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59938f3-ace7-414d-924f-5991f3eabf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:55.858072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59938f3-ace7-414d-924f-5991f3eabf77 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:55.858894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59938f3-ace7-414d-924f-5991f3eabf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:55.859171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59938f3-ace7-414d-924f-5991f3eabf77 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:55.859846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59938f3-ace7-414d-924f-5991f3eabf77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:55.861830 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59938f3-ace7-414d-924f-5991f3eabf77 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:55.960090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1114/2234] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:43:55 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:43:55.964841 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09367e-f1bb-40a1-ad9d-502b5c3a582d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:55.976891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09367e-f1bb-40a1-ad9d-502b5c3a582d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:55.986422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879634-5fd9-4020-a7c9-a678d32b5d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:55.986422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879634-5fd9-4020-a7c9-a678d32b5d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:55.986422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879634-5fd9-4020-a7c9-a678d32b5d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:55.986422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879634-5fd9-4020-a7c9-a678d32b5d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:55.987821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe879634-5fd9-4020-a7c9-a678d32b5d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:55.990414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe879634-5fd9-4020-a7c9-a678d32b5d02 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:56.008953 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1121/2235] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:56.015291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59938f3-ace7-414d-924f-5991f3eabf77 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:56.053201 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59938f3-ace7-414d-924f-5991f3eabf77 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:56.072498 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a0659-294b-43c8-98cc-dd4a0b3d37c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:56.073319 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a0659-294b-43c8-98cc-dd4a0b3d37c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:56.073612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a0659-294b-43c8-98cc-dd4a0b3d37c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:56.074009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a0659-294b-43c8-98cc-dd4a0b3d37c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:56.074609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a0659-294b-43c8-98cc-dd4a0b3d37c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:56.076350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a0659-294b-43c8-98cc-dd4a0b3d37c2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:56.095826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1115/2236] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:43:56.104963 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879634-5fd9-4020-a7c9-a678d32b5d02 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:56.116341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879634-5fd9-4020-a7c9-a678d32b5d02 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:56.121292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b2a1a-2754-4203-8d99-c4cb3438753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:56.124210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b2a1a-2754-4203-8d99-c4cb3438753d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:56.124210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b2a1a-2754-4203-8d99-c4cb3438753d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:56.124210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b2a1a-2754-4203-8d99-c4cb3438753d admin admin] PATH_INFO: `/v3/projects/36e7dc1d31904ac6ab620ca03e9f80ac/users/07e4744a1b354d528419376abae17ad5/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:56.138744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b2a1a-2754-4203-8d99-c4cb3438753d admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e7dc1d31904ac6ab620ca03e9f80ac, user_id=07e4744a1b354d528419376abae17ad5, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:56.142058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b2a1a-2754-4203-8d99-c4cb3438753d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:56.277669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1122/2237] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:43:56 2022] PUT /identity/v3/projects/36e7dc1d31904ac6ab620ca03e9f80ac/users/07e4744a1b354d528419376abae17ad5/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:43:56.287996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a9828-d338-49b9-948d-450d4afff9c3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:56.288321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a9828-d338-49b9-948d-450d4afff9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:56.288618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a9828-d338-49b9-948d-450d4afff9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:56.357199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-181a9828-d338-49b9-948d-450d4afff9c3 None None] MFA Rules not processed for user `07e4744a1b354d528419376abae17ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:43:56.359356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-181a9828-d338-49b9-948d-450d4afff9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:56.404830 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:43:56.450238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1116/2238] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:43:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:43:56.462317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b2a1a-2754-4203-8d99-c4cb3438753d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:43:56.464629 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ac3b2a1a-2754-4203-8d99-c4cb3438753d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:43:56.525473 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b2a1a-2754-4203-8d99-c4cb3438753d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:43:56.529120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037d7df-fc36-48b8-8324-34f057063182 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfc4dfda424d40b7c9f17345b8ad45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:43:56.529729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037d7df-fc36-48b8-8324-34f057063182 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:43:56.530009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037d7df-fc36-48b8-8324-34f057063182 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:43:56.530330 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037d7df-fc36-48b8-8324-34f057063182 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:43:56.539240 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037d7df-fc36-48b8-8324-34f057063182 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:43:56.540753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037d7df-fc36-48b8-8324-34f057063182 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:43:56.550169 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1123/2239] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:43:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:44:06.529369 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=17 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:06.545644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:06.908668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9cf03-b4a5-4ffc-ab04-2ae9d370b9c5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:06.908961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9cf03-b4a5-4ffc-ab04-2ae9d370b9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:06.909933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9cf03-b4a5-4ffc-ab04-2ae9d370b9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:06.929033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-29f9cf03-b4a5-4ffc-ab04-2ae9d370b9c5 None None] MFA Rules not processed for user `88d78d9c0369431d88ad85c960f6a0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:06.930250 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-29f9cf03-b4a5-4ffc-ab04-2ae9d370b9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:06.987944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1117/2240] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:44:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:44:16.980988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=25 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:26.984392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6d2da-d899-4736-8f4f-948fa8b5df58 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:26.984877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6d2da-d899-4736-8f4f-948fa8b5df58 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:26.984982 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6d2da-d899-4736-8f4f-948fa8b5df58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:27.004255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e9b6d2da-d899-4736-8f4f-948fa8b5df58 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:27.005455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e9b6d2da-d899-4736-8f4f-948fa8b5df58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:27.064192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1124/2241] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:44:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:44:27.111348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9cf03-b4a5-4ffc-ab04-2ae9d370b9c5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:27.112872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-29f9cf03-b4a5-4ffc-ab04-2ae9d370b9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:27.190853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9cf03-b4a5-4ffc-ab04-2ae9d370b9c5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:27.196059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3410d8-0b2b-4bf3-8b92-8e4cea5e6aa6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:27.196702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3410d8-0b2b-4bf3-8b92-8e4cea5e6aa6 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:27.196989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3410d8-0b2b-4bf3-8b92-8e4cea5e6aa6 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:27.197269 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3410d8-0b2b-4bf3-8b92-8e4cea5e6aa6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:27.206797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3410d8-0b2b-4bf3-8b92-8e4cea5e6aa6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:27.208445 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3410d8-0b2b-4bf3-8b92-8e4cea5e6aa6 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:27.218743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1118/2242] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:44:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:44:32.019483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6d2da-d899-4736-8f4f-948fa8b5df58 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:32.164820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6d2da-d899-4736-8f4f-948fa8b5df58 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:32.177900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:32.178744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:32.179115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:32.179378 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] PATH_INFO: `/v3/users/07e4744a1b354d528419376abae17ad5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:32.180895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e4744a1b354d528419376abae17ad5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:32.185659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:32.471864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e4744a1b354d528419376abae17ad5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:32.479071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e4744a1b354d528419376abae17ad5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:32.490097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e4744a1b354d528419376abae17ad5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:32.517418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e4744a1b354d528419376abae17ad5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:32.616437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e4744a1b354d528419376abae17ad5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:32.625838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1125/2243] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:44:32 2022] DELETE /identity/v3/users/07e4744a1b354d528419376abae17ad5 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:44:32.630472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d72dfa-4710-4101-8c70-4a10f141f539 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:32.630996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d72dfa-4710-4101-8c70-4a10f141f539 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:32.631238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d72dfa-4710-4101-8c70-4a10f141f539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:32.653137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-13d72dfa-4710-4101-8c70-4a10f141f539 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:32.655643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-13d72dfa-4710-4101-8c70-4a10f141f539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:32.745567 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1119/2244] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:44:32.755370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:32.756939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:32.799506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:32.863644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae064d6c-6349-45cb-875d-fbdda1888dcf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:32.868977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62451fc-b33e-47d3-928d-9fc5753e3491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:32.869669 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62451fc-b33e-47d3-928d-9fc5753e3491 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:32.869955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62451fc-b33e-47d3-928d-9fc5753e3491 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:32.870238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62451fc-b33e-47d3-928d-9fc5753e3491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:32.883705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62451fc-b33e-47d3-928d-9fc5753e3491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:32.886036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62451fc-b33e-47d3-928d-9fc5753e3491 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:32.897285 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1126/2245] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:44:33.485783 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d72dfa-4710-4101-8c70-4a10f141f539 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:33.487629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-13d72dfa-4710-4101-8c70-4a10f141f539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:33.504390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d72dfa-4710-4101-8c70-4a10f141f539 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:33.511581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:33.512313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:33.512595 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:33.512870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] PATH_INFO: `/v3/projects/36e7dc1d31904ac6ab620ca03e9f80ac` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:33.515296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36e7dc1d31904ac6ab620ca03e9f80ac)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:33.516978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:33.719604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e7dc1d31904ac6ab620ca03e9f80ac', 'request_id': 'req-4f153570-49e9-4990-8341-f211e6139d12'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:33.728180 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e7dc1d31904ac6ab620ca03e9f80ac', 'request_id': 'req-4f153570-49e9-4990-8341-f211e6139d12'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:33.809664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e7dc1d31904ac6ab620ca03e9f80ac', 'request_id': 'req-4f153570-49e9-4990-8341-f211e6139d12'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:33.823006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] The token cache is being invalidate because project 36e7dc1d31904ac6ab620ca03e9f80ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:44:33.823304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:33.824043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f153570-49e9-4990-8341-f211e6139d12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:33.826015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1120/2246] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:44:33 2022] DELETE /identity/v3/projects/36e7dc1d31904ac6ab620ca03e9f80ac => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:44:33.833525 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a218f-4c0a-4de6-8f66-8205d9892d4d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:33.834042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a218f-4c0a-4de6-8f66-8205d9892d4d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:33.834538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a218f-4c0a-4de6-8f66-8205d9892d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:33.862589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b60a218f-4c0a-4de6-8f66-8205d9892d4d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:33.864274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b60a218f-4c0a-4de6-8f66-8205d9892d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:33.949738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1127/2247] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:44:33.951295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2765695-cb6b-4291-8920-b0bf84155138 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:33.951751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2765695-cb6b-4291-8920-b0bf84155138 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:33.952133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2765695-cb6b-4291-8920-b0bf84155138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:33.980494 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a2765695-cb6b-4291-8920-b0bf84155138 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:33.981965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a2765695-cb6b-4291-8920-b0bf84155138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:33.990789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1121/2248] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:44:33.995095 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a218f-4c0a-4de6-8f66-8205d9892d4d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:34.009154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a218f-4c0a-4de6-8f66-8205d9892d4d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:34.015073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a55de-389d-4794-93f6-5fdeffbddf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:34.015729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a55de-389d-4794-93f6-5fdeffbddf18 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:34.016038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a55de-389d-4794-93f6-5fdeffbddf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:34.016319 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a55de-389d-4794-93f6-5fdeffbddf18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:34.019106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a55de-389d-4794-93f6-5fdeffbddf18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:44:34.019423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a55de-389d-4794-93f6-5fdeffbddf18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:34.021091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a55de-389d-4794-93f6-5fdeffbddf18 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:34.036053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1128/2249] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:44:34.041769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2765695-cb6b-4291-8920-b0bf84155138 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:34.052509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2765695-cb6b-4291-8920-b0bf84155138 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:34.058869 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f3ad5-4123-43ee-aae5-0cc2542a85ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:34.059387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f3ad5-4123-43ee-aae5-0cc2542a85ae admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:34.059675 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f3ad5-4123-43ee-aae5-0cc2542a85ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:34.060439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f3ad5-4123-43ee-aae5-0cc2542a85ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:34.061696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f3ad5-4123-43ee-aae5-0cc2542a85ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:34.063640 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f3ad5-4123-43ee-aae5-0cc2542a85ae admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:34.068443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:34.190161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1122/2250] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:44:34 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:44:34.195733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a55de-389d-4794-93f6-5fdeffbddf18 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:34.202376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a55de-389d-4794-93f6-5fdeffbddf18 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:34.274929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985666c-a947-45cd-85aa-0c6365f827eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:34.275665 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985666c-a947-45cd-85aa-0c6365f827eb admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:34.275949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985666c-a947-45cd-85aa-0c6365f827eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:34.276271 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985666c-a947-45cd-85aa-0c6365f827eb admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:34.277009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985666c-a947-45cd-85aa-0c6365f827eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:34.281222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985666c-a947-45cd-85aa-0c6365f827eb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:34.376390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1129/2251] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:44:34 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:44:34.377797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f3ad5-4123-43ee-aae5-0cc2542a85ae admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:34.389256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f3ad5-4123-43ee-aae5-0cc2542a85ae admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:34.394210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb9841-d19b-4512-a222-438d9c34f42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:34.394957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb9841-d19b-4512-a222-438d9c34f42b admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:34.395351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb9841-d19b-4512-a222-438d9c34f42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:34.395699 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb9841-d19b-4512-a222-438d9c34f42b admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:34.398570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb9841-d19b-4512-a222-438d9c34f42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:34.398570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb9841-d19b-4512-a222-438d9c34f42b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:34.410296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1123/2252] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:44:34.415234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985666c-a947-45cd-85aa-0c6365f827eb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:34.427582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985666c-a947-45cd-85aa-0c6365f827eb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:34.434379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce2042-9a41-4e53-9cef-b290871d2df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:34.436062 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce2042-9a41-4e53-9cef-b290871d2df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:34.436484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce2042-9a41-4e53-9cef-b290871d2df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:34.436759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce2042-9a41-4e53-9cef-b290871d2df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:34.437454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce2042-9a41-4e53-9cef-b290871d2df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:34.439084 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce2042-9a41-4e53-9cef-b290871d2df9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:34.455259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1130/2253] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:44:34.456474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb9841-d19b-4512-a222-438d9c34f42b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:34.467020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb9841-d19b-4512-a222-438d9c34f42b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:34.475490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b1e4c-1f40-4c18-9f7f-f5b159064a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:34.476714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b1e4c-1f40-4c18-9f7f-f5b159064a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:34.476997 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b1e4c-1f40-4c18-9f7f-f5b159064a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:34.477272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b1e4c-1f40-4c18-9f7f-f5b159064a66 admin admin] PATH_INFO: `/v3/projects/39975b18c0de40feb307036d53a8f70c/users/5da03ec05e1f4e5f96c7d38c88c81b65/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:34.496684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b1e4c-1f40-4c18-9f7f-f5b159064a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39975b18c0de40feb307036d53a8f70c, user_id=5da03ec05e1f4e5f96c7d38c88c81b65, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:34.500038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b1e4c-1f40-4c18-9f7f-f5b159064a66 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:34.605049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1124/2254] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:44:34 2022] PUT /identity/v3/projects/39975b18c0de40feb307036d53a8f70c/users/5da03ec05e1f4e5f96c7d38c88c81b65/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:44:34.615373 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8ef3a-b8ff-4f0e-b157-a43e88d9baa4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:34.615658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8ef3a-b8ff-4f0e-b157-a43e88d9baa4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:34.615939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8ef3a-b8ff-4f0e-b157-a43e88d9baa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:34.655976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4ed8ef3a-b8ff-4f0e-b157-a43e88d9baa4 None None] MFA Rules not processed for user `5da03ec05e1f4e5f96c7d38c88c81b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:34.661451 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4ed8ef3a-b8ff-4f0e-b157-a43e88d9baa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:34.690360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:34.743748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1131/2255] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:44:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:44:34.757097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b1e4c-1f40-4c18-9f7f-f5b159064a66 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:34.760500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-511b1e4c-1f40-4c18-9f7f-f5b159064a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:34.846556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b1e4c-1f40-4c18-9f7f-f5b159064a66 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:34.860711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c95f8-7adb-46f8-a612-d98536e721a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:34.862942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c95f8-7adb-46f8-a612-d98536e721a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:34.862942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c95f8-7adb-46f8-a612-d98536e721a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:34.862942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c95f8-7adb-46f8-a612-d98536e721a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:34.890319 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c95f8-7adb-46f8-a612-d98536e721a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:34.895921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c95f8-7adb-46f8-a612-d98536e721a8 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:34.909853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1125/2256] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:44:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:44:34.955115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8ef3a-b8ff-4f0e-b157-a43e88d9baa4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:35.067174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8ef3a-b8ff-4f0e-b157-a43e88d9baa4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:35.085084 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:35.085838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:35.086269 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:35.086615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] PATH_INFO: `/v3/users/5da03ec05e1f4e5f96c7d38c88c81b65` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:35.088104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da03ec05e1f4e5f96c7d38c88c81b65)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:35.090590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:35.288490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da03ec05e1f4e5f96c7d38c88c81b65'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:35.289528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da03ec05e1f4e5f96c7d38c88c81b65'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:35.298114 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da03ec05e1f4e5f96c7d38c88c81b65'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:35.315674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da03ec05e1f4e5f96c7d38c88c81b65'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:35.431475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da03ec05e1f4e5f96c7d38c88c81b65'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:35.438817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1132/2257] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:44:34 2022] DELETE /identity/v3/users/5da03ec05e1f4e5f96c7d38c88c81b65 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:44:35.445961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f07ea-a2b3-4129-a050-57688457574f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:35.445961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f07ea-a2b3-4129-a050-57688457574f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:35.445961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f07ea-a2b3-4129-a050-57688457574f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:35.470290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a98f07ea-a2b3-4129-a050-57688457574f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:35.471543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a98f07ea-a2b3-4129-a050-57688457574f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:35.559594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1126/2258] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:44:35.573156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:35.573156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:35.633095 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:35.650180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b9f46-b71c-436f-b141-e21d16d01ade admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:35.654533 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10e2bd-5b2d-4416-9583-510939a857df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:35.655197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10e2bd-5b2d-4416-9583-510939a857df service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:35.655481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10e2bd-5b2d-4416-9583-510939a857df service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:35.656016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10e2bd-5b2d-4416-9583-510939a857df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:35.667452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10e2bd-5b2d-4416-9583-510939a857df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:35.668959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10e2bd-5b2d-4416-9583-510939a857df service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:35.695034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1133/2259] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:44:36.182838 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f07ea-a2b3-4129-a050-57688457574f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:36.185327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a98f07ea-a2b3-4129-a050-57688457574f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:36.216788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f07ea-a2b3-4129-a050-57688457574f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:36.218267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:36.218988 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:36.222527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:36.223450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] PATH_INFO: `/v3/projects/39975b18c0de40feb307036d53a8f70c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:36.236033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39975b18c0de40feb307036d53a8f70c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:36.245621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:36.402802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:36.404965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39975b18c0de40feb307036d53a8f70c', 'request_id': 'req-8f0edf57-6b65-4737-bc82-542d0d6b5540'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:36.411090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39975b18c0de40feb307036d53a8f70c', 'request_id': 'req-8f0edf57-6b65-4737-bc82-542d0d6b5540'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:36.508902 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39975b18c0de40feb307036d53a8f70c', 'request_id': 'req-8f0edf57-6b65-4737-bc82-542d0d6b5540'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:36.525683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] The token cache is being invalidate because project 39975b18c0de40feb307036d53a8f70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:44:36.526090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:36.527083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:36.529802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1127/2260] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:44:36 2022] DELETE /identity/v3/projects/39975b18c0de40feb307036d53a8f70c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:44:43.993444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611c54-3e04-4a60-a7c6-5ba00fed2820 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:43.993949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611c54-3e04-4a60-a7c6-5ba00fed2820 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:43.994052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611c54-3e04-4a60-a7c6-5ba00fed2820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:44.014456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-77611c54-3e04-4a60-a7c6-5ba00fed2820 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:44.015578 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-77611c54-3e04-4a60-a7c6-5ba00fed2820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:44.070056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1134/2261] 10.222.0.64 () {58 vars in 967 bytes} [Thu Aug 4 07:44:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:44:44.082870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:44.084290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:44.159010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0edf57-6b65-4737-bc82-542d0d6b5540 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:44.163946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343020fe-faab-4306-9c14-2f3bf72dd0b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:44.164591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343020fe-faab-4306-9c14-2f3bf72dd0b6 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:44.164877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343020fe-faab-4306-9c14-2f3bf72dd0b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:44.165160 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343020fe-faab-4306-9c14-2f3bf72dd0b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:44.173852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343020fe-faab-4306-9c14-2f3bf72dd0b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:44.175474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343020fe-faab-4306-9c14-2f3bf72dd0b6 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:44.183470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1128/2262] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:44:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:44:47.490059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77611c54-3e04-4a60-a7c6-5ba00fed2820 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:47.491562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-77611c54-3e04-4a60-a7c6-5ba00fed2820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:47.562024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77611c54-3e04-4a60-a7c6-5ba00fed2820 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:47.566957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1a123-6150-4650-b6c4-3d077948d339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:47.567619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a123-6150-4650-b6c4-3d077948d339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:47.567900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a123-6150-4650-b6c4-3d077948d339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:47.568206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a123-6150-4650-b6c4-3d077948d339 admin admin] PATH_INFO: `/v3/users/b882f6093b6743dabd6c964b086842b0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:47.569379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1a123-6150-4650-b6c4-3d077948d339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b882f6093b6743dabd6c964b086842b0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:47.571312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1a123-6150-4650-b6c4-3d077948d339 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:47.829075 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dfe1a123-6150-4650-b6c4-3d077948d339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b882f6093b6743dabd6c964b086842b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:47.829725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dfe1a123-6150-4650-b6c4-3d077948d339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b882f6093b6743dabd6c964b086842b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:47.836997 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dfe1a123-6150-4650-b6c4-3d077948d339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b882f6093b6743dabd6c964b086842b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:47.849127 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dfe1a123-6150-4650-b6c4-3d077948d339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b882f6093b6743dabd6c964b086842b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:47.976921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dfe1a123-6150-4650-b6c4-3d077948d339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b882f6093b6743dabd6c964b086842b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:47.983423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1135/2263] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:44:47 2022] DELETE /identity/v3/users/b882f6093b6743dabd6c964b086842b0 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:44:48.232694 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343020fe-faab-4306-9c14-2f3bf72dd0b6 service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:48.245648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343020fe-faab-4306-9c14-2f3bf72dd0b6 service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:48.307154 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:48.307800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:48.308115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:48.308434 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] PATH_INFO: `/v3/projects/8d037ddd73f14dda82de565f8530d9ee` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:48.309581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d037ddd73f14dda82de565f8530d9ee)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:48.311486 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:48.445876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d037ddd73f14dda82de565f8530d9ee', 'request_id': 'req-192f47e9-189e-48a3-9b5b-1e28e6933db7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:48.452240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d037ddd73f14dda82de565f8530d9ee', 'request_id': 'req-192f47e9-189e-48a3-9b5b-1e28e6933db7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:48.545049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d037ddd73f14dda82de565f8530d9ee', 'request_id': 'req-192f47e9-189e-48a3-9b5b-1e28e6933db7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:48.550752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:48.556747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] The token cache is being invalidate because project 8d037ddd73f14dda82de565f8530d9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:44:48.557039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:48.557673 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-192f47e9-189e-48a3-9b5b-1e28e6933db7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:44:48.559201 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1129/2264] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:44:48 2022] DELETE /identity/v3/projects/8d037ddd73f14dda82de565f8530d9ee => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:44:48.566231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ade4f-1c0f-47fe-87f6-dc330060192c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:48.566730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ade4f-1c0f-47fe-87f6-dc330060192c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:48.566825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ade4f-1c0f-47fe-87f6-dc330060192c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:48.586951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5a4ade4f-1c0f-47fe-87f6-dc330060192c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:48.588429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5a4ade4f-1c0f-47fe-87f6-dc330060192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:48.592568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:48.647118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1136/2265] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:44:48.655737 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066f06c-0356-4d26-9b99-e35196a001b8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:48.657642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066f06c-0356-4d26-9b99-e35196a001b8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:48.657954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066f06c-0356-4d26-9b99-e35196a001b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:48.678046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b066f06c-0356-4d26-9b99-e35196a001b8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:48.679274 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b066f06c-0356-4d26-9b99-e35196a001b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:48.687550 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1130/2266] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:44:48.692253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ade4f-1c0f-47fe-87f6-dc330060192c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:48.702021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ade4f-1c0f-47fe-87f6-dc330060192c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:48.706811 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177ae43-3aba-45ac-823e-fe15bd120e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:48.707414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177ae43-3aba-45ac-823e-fe15bd120e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:48.707696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177ae43-3aba-45ac-823e-fe15bd120e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:48.707971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177ae43-3aba-45ac-823e-fe15bd120e14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:48.708886 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177ae43-3aba-45ac-823e-fe15bd120e14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:44:48.709190 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177ae43-3aba-45ac-823e-fe15bd120e14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:48.710777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177ae43-3aba-45ac-823e-fe15bd120e14 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:48.723498 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1137/2267] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:44:48.729700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066f06c-0356-4d26-9b99-e35196a001b8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:48.740412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066f06c-0356-4d26-9b99-e35196a001b8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:48.745199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13468c4-fcde-413c-8501-b086d5257d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:48.745805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13468c4-fcde-413c-8501-b086d5257d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:48.746117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13468c4-fcde-413c-8501-b086d5257d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:48.746394 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13468c4-fcde-413c-8501-b086d5257d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:48.747102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13468c4-fcde-413c-8501-b086d5257d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:48.749106 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13468c4-fcde-413c-8501-b086d5257d60 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:48.885010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1131/2268] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:44:48 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:44:48.888370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177ae43-3aba-45ac-823e-fe15bd120e14 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:48.898822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177ae43-3aba-45ac-823e-fe15bd120e14 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:48.929283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13468c4-fcde-413c-8501-b086d5257d60 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:48.930996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b13468c4-fcde-413c-8501-b086d5257d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:48.960864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e657bc-0e75-4323-a5c8-dfb84e3788f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:48.961448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e657bc-0e75-4323-a5c8-dfb84e3788f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:48.961730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e657bc-0e75-4323-a5c8-dfb84e3788f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:48.962147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e657bc-0e75-4323-a5c8-dfb84e3788f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:48.962878 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e657bc-0e75-4323-a5c8-dfb84e3788f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:48.964990 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e657bc-0e75-4323-a5c8-dfb84e3788f6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:49.000264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13468c4-fcde-413c-8501-b086d5257d60 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:49.005363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32abff-864f-409e-8c46-dcf6fc10ea9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:49.005982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32abff-864f-409e-8c46-dcf6fc10ea9b service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:49.006264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32abff-864f-409e-8c46-dcf6fc10ea9b service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:49.006539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32abff-864f-409e-8c46-dcf6fc10ea9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:49.008318 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9c32abff-864f-409e-8c46-dcf6fc10ea9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:49.019968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32abff-864f-409e-8c46-dcf6fc10ea9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:49.021595 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32abff-864f-409e-8c46-dcf6fc10ea9b service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:49.032066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1132/2269] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:44:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:44:49.105967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1138/2270] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:44:48 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:44:49.110040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32abff-864f-409e-8c46-dcf6fc10ea9b service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:49.120618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32abff-864f-409e-8c46-dcf6fc10ea9b service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:49.125262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a2bde-3ef1-4cfa-8676-2b050a256b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:49.125874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2bde-3ef1-4cfa-8676-2b050a256b30 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:49.126159 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2bde-3ef1-4cfa-8676-2b050a256b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:49.126502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2bde-3ef1-4cfa-8676-2b050a256b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:49.127209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a2bde-3ef1-4cfa-8676-2b050a256b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:49.128869 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a2bde-3ef1-4cfa-8676-2b050a256b30 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:49.138951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1133/2271] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:44:49.163757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e657bc-0e75-4323-a5c8-dfb84e3788f6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:49.173978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e657bc-0e75-4323-a5c8-dfb84e3788f6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:49.178757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797f2a3-087e-4cf0-991f-e66fd03adaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:49.179377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797f2a3-087e-4cf0-991f-e66fd03adaa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:49.179660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797f2a3-087e-4cf0-991f-e66fd03adaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:49.179933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797f2a3-087e-4cf0-991f-e66fd03adaa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:49.180714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797f2a3-087e-4cf0-991f-e66fd03adaa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:49.182372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797f2a3-087e-4cf0-991f-e66fd03adaa8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:49.193202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1139/2272] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:44:49.202048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a2bde-3ef1-4cfa-8676-2b050a256b30 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:49.212713 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a2bde-3ef1-4cfa-8676-2b050a256b30 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:49.217282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a10fb-c094-4a97-ae19-cbf4d0ab76fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:49.217936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a10fb-c094-4a97-ae19-cbf4d0ab76fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:49.218220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a10fb-c094-4a97-ae19-cbf4d0ab76fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:49.219055 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a10fb-c094-4a97-ae19-cbf4d0ab76fc admin admin] PATH_INFO: `/v3/projects/e5947c5779f74cc396cf2146a32e8724/users/4588b8c597ba431cbf2e350ed218673d/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:49.233922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a10fb-c094-4a97-ae19-cbf4d0ab76fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5947c5779f74cc396cf2146a32e8724, user_id=4588b8c597ba431cbf2e350ed218673d, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:49.236803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a10fb-c094-4a97-ae19-cbf4d0ab76fc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:49.380465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1134/2273] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:44:49 2022] PUT /identity/v3/projects/e5947c5779f74cc396cf2146a32e8724/users/4588b8c597ba431cbf2e350ed218673d/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:44:49.391644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8ad26-23ad-4835-a5fa-9665f8496248 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:49.392176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8ad26-23ad-4835-a5fa-9665f8496248 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:49.392453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8ad26-23ad-4835-a5fa-9665f8496248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:49.422398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-caa8ad26-23ad-4835-a5fa-9665f8496248 None None] MFA Rules not processed for user `4588b8c597ba431cbf2e350ed218673d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:49.423735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-caa8ad26-23ad-4835-a5fa-9665f8496248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:49.490733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1140/2274] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:44:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:44:49.504602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a10fb-c094-4a97-ae19-cbf4d0ab76fc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:49.506301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0f4a10fb-c094-4a97-ae19-cbf4d0ab76fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:49.565873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a10fb-c094-4a97-ae19-cbf4d0ab76fc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:49.569638 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55456fa-ec48-40a0-a566-606abb93dcbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:49.570305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55456fa-ec48-40a0-a566-606abb93dcbd service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:49.570593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55456fa-ec48-40a0-a566-606abb93dcbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:49.570872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55456fa-ec48-40a0-a566-606abb93dcbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:49.580472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55456fa-ec48-40a0-a566-606abb93dcbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:49.581928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55456fa-ec48-40a0-a566-606abb93dcbd service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:49.590914 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1135/2275] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:44:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:44:54.786858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67220f-de27-45b7-bfe4-83e62a45da81 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:54.787318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67220f-de27-45b7-bfe4-83e62a45da81 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:54.787433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67220f-de27-45b7-bfe4-83e62a45da81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:54.795501 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:54.805958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ed67220f-de27-45b7-bfe4-83e62a45da81 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:44:54.807150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ed67220f-de27-45b7-bfe4-83e62a45da81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:54.861625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1141/2276] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:44:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:44:54.873520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55456fa-ec48-40a0-a566-606abb93dcbd service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:44:54.875502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d55456fa-ec48-40a0-a566-606abb93dcbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:44:54.905832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:44:54.947607 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55456fa-ec48-40a0-a566-606abb93dcbd service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:44:54.953836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f7626-edc6-4066-8d23-5344371812b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:44:54.954464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7626-edc6-4066-8d23-5344371812b3 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:44:54.954843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7626-edc6-4066-8d23-5344371812b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:44:54.955172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7626-edc6-4066-8d23-5344371812b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:44:54.964549 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f7626-edc6-4066-8d23-5344371812b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:44:54.966242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f7626-edc6-4066-8d23-5344371812b3 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:44:54.974862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1136/2277] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:44:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:45:04.885545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=23 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:45:05.111020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=21 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:45:14.837090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa86eed-220e-4b43-a0f9-6cf862187f92 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:14.837565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa86eed-220e-4b43-a0f9-6cf862187f92 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:14.842130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa86eed-220e-4b43-a0f9-6cf862187f92 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:14.842130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1142/2278] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:45:14.846594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5370d4-9be2-4bfe-97a1-ec5c5edd834c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:14.847053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5370d4-9be2-4bfe-97a1-ec5c5edd834c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:14.847275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5370d4-9be2-4bfe-97a1-ec5c5edd834c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:14.870906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2b5370d4-9be2-4bfe-97a1-ec5c5edd834c None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:45:14.872226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2b5370d4-9be2-4bfe-97a1-ec5c5edd834c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:14.945110 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1137/2279] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:45:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:45:14.964518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa86eed-220e-4b43-a0f9-6cf862187f92 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:45:14.966381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-afa86eed-220e-4b43-a0f9-6cf862187f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:15.034529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa86eed-220e-4b43-a0f9-6cf862187f92 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:45:15.038073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c006800-f02e-49d5-a370-1b1a2ebbd68f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfc4dfda424d40b7c9f17345b8ad45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:45:15.038818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c006800-f02e-49d5-a370-1b1a2ebbd68f service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:15.039127 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c006800-f02e-49d5-a370-1b1a2ebbd68f service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:15.039411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c006800-f02e-49d5-a370-1b1a2ebbd68f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:15.051740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c006800-f02e-49d5-a370-1b1a2ebbd68f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:45:15.053225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c006800-f02e-49d5-a370-1b1a2ebbd68f service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:45:15.064551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1143/2280] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:45:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:45:15.284476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cc972-08b4-43ac-8bd0-065e1455f235 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:15.284769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cc972-08b4-43ac-8bd0-065e1455f235 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:15.285049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cc972-08b4-43ac-8bd0-065e1455f235 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:15.286403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1138/2281] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:45:15.291528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17546d-5b7d-4f1c-a9b9-53ea91d8b040 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:15.291946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17546d-5b7d-4f1c-a9b9-53ea91d8b040 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:15.292218 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17546d-5b7d-4f1c-a9b9-53ea91d8b040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:15.313098 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8f17546d-5b7d-4f1c-a9b9-53ea91d8b040 None None] MFA Rules not processed for user `b80e701a81cb4fa893e55288559010f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:45:15.314289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8f17546d-5b7d-4f1c-a9b9-53ea91d8b040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:15.374293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1144/2282] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:45:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:45:15.387513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cc972-08b4-43ac-8bd0-065e1455f235 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:45:15.390166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-631cc972-08b4-43ac-8bd0-065e1455f235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:15.405808 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cc972-08b4-43ac-8bd0-065e1455f235 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:45:15.409957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69496747-9145-47a2-a1fa-46508d14f2bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:45:15.410571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69496747-9145-47a2-a1fa-46508d14f2bb service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:15.410850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69496747-9145-47a2-a1fa-46508d14f2bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:15.411120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69496747-9145-47a2-a1fa-46508d14f2bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:15.418500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69496747-9145-47a2-a1fa-46508d14f2bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:45:15.420043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69496747-9145-47a2-a1fa-46508d14f2bb service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:45:15.429653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1139/2283] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:45:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:45:15.516639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d37466-8ecd-4426-86bf-9789498b2906 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:15.517017 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d37466-8ecd-4426-86bf-9789498b2906 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:15.517301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d37466-8ecd-4426-86bf-9789498b2906 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:15.518678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1145/2284] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:45:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:45:15.524186 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6c4a-021b-4ec8-be77-bf28ef43ed15 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:15.524558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6c4a-021b-4ec8-be77-bf28ef43ed15 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:15.524900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6c4a-021b-4ec8-be77-bf28ef43ed15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:15.545903 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-378a6c4a-021b-4ec8-be77-bf28ef43ed15 None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:45:15.547146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-378a6c4a-021b-4ec8-be77-bf28ef43ed15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:15.558422 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1140/2285] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:45:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:45:15.569597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d37466-8ecd-4426-86bf-9789498b2906 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:45:15.571368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-45d37466-8ecd-4426-86bf-9789498b2906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:15.586589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d37466-8ecd-4426-86bf-9789498b2906 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:45:15.590727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e2471-3aef-4040-b0d6-bd3cf94ec49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:45:15.591413 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e2471-3aef-4040-b0d6-bd3cf94ec49f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:15.591748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e2471-3aef-4040-b0d6-bd3cf94ec49f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:15.592115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e2471-3aef-4040-b0d6-bd3cf94ec49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:15.603353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e2471-3aef-4040-b0d6-bd3cf94ec49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:45:15.604821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e2471-3aef-4040-b0d6-bd3cf94ec49f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:45:15.615910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1146/2286] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:45:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:45:21.151497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e386dbd-3187-47cd-b41f-5b351b7c4aac None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:21.152032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e386dbd-3187-47cd-b41f-5b351b7c4aac None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:21.152253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e386dbd-3187-47cd-b41f-5b351b7c4aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:21.185238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1e386dbd-3187-47cd-b41f-5b351b7c4aac None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:45:21.186747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1e386dbd-3187-47cd-b41f-5b351b7c4aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:21.201118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1141/2287] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:45:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:45:21.216036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e2471-3aef-4040-b0d6-bd3cf94ec49f service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:45:21.237461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e2471-3aef-4040-b0d6-bd3cf94ec49f service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:45:21.242537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb087e3-d891-4d2a-a479-12910dae15c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:45:21.243172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb087e3-d891-4d2a-a479-12910dae15c3 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:21.243459 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb087e3-d891-4d2a-a479-12910dae15c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:21.243722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb087e3-d891-4d2a-a479-12910dae15c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:21.252823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb087e3-d891-4d2a-a479-12910dae15c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:45:21.254478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb087e3-d891-4d2a-a479-12910dae15c3 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:45:21.264071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1147/2288] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:45:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:45:31.173131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=43 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:45:31.260559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=61 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:45:33.160799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e386dbd-3187-47cd-b41f-5b351b7c4aac None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:45:33.245633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e386dbd-3187-47cd-b41f-5b351b7c4aac None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:45:33.250376 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:45:33.251134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:33.251532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:33.252278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] PATH_INFO: `/v3/users/4588b8c597ba431cbf2e350ed218673d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:33.255107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4588b8c597ba431cbf2e350ed218673d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:45:33.255107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:45:33.448984 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4588b8c597ba431cbf2e350ed218673d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:45:33.449640 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4588b8c597ba431cbf2e350ed218673d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:45:33.450244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4588b8c597ba431cbf2e350ed218673d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:45:33.463053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4588b8c597ba431cbf2e350ed218673d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:45:33.538493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4588b8c597ba431cbf2e350ed218673d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:45:33.551446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1142/2289] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:45:33 2022] DELETE /identity/v3/users/4588b8c597ba431cbf2e350ed218673d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:45:33.555908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c745f-cf03-4b2b-bb9a-a9c0aa5f8130 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:33.556446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c745f-cf03-4b2b-bb9a-a9c0aa5f8130 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:33.556547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c745f-cf03-4b2b-bb9a-a9c0aa5f8130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:33.575950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-514c745f-cf03-4b2b-bb9a-a9c0aa5f8130 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:45:33.577203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-514c745f-cf03-4b2b-bb9a-a9c0aa5f8130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:33.638582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1148/2290] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:45:33.649007 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:45:33.653043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:33.710056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abb457-a353-4620-9078-ae2ee8137672 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:45:33.713934 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcb717-bbdc-439b-b8d3-060f38f9a52d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:45:33.714556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcb717-bbdc-439b-b8d3-060f38f9a52d service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:33.714840 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcb717-bbdc-439b-b8d3-060f38f9a52d service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:33.715113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcb717-bbdc-439b-b8d3-060f38f9a52d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:33.726137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcb717-bbdc-439b-b8d3-060f38f9a52d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:45:33.727609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcb717-bbdc-439b-b8d3-060f38f9a52d service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:45:33.738470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1143/2291] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:45:35.033102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c745f-cf03-4b2b-bb9a-a9c0aa5f8130 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:45:35.035261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-514c745f-cf03-4b2b-bb9a-a9c0aa5f8130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:35.048554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c745f-cf03-4b2b-bb9a-a9c0aa5f8130 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:45:35.053867 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:45:35.054455 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:35.054754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:35.055024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] PATH_INFO: `/v3/projects/e5947c5779f74cc396cf2146a32e8724` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:35.056401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5947c5779f74cc396cf2146a32e8724)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:45:35.058549 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:45:35.194021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5947c5779f74cc396cf2146a32e8724', 'request_id': 'req-1c5b33d3-ee77-4a5f-a54d-a588faf27180'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:45:35.301368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5947c5779f74cc396cf2146a32e8724', 'request_id': 'req-1c5b33d3-ee77-4a5f-a54d-a588faf27180'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:45:35.307345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5947c5779f74cc396cf2146a32e8724', 'request_id': 'req-1c5b33d3-ee77-4a5f-a54d-a588faf27180'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:45:35.318462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] The token cache is being invalidate because project e5947c5779f74cc396cf2146a32e8724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:45:35.318753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:45:35.319385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1c5b33d3-ee77-4a5f-a54d-a588faf27180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:45:35.320931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1149/2292] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:45:35 2022] DELETE /identity/v3/projects/e5947c5779f74cc396cf2146a32e8724 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:45:36.558169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb037dd-c249-414d-94ad-403bcc0381ba None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:36.558669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb037dd-c249-414d-94ad-403bcc0381ba None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:36.558761 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb037dd-c249-414d-94ad-403bcc0381ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:36.578965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dcb037dd-c249-414d-94ad-403bcc0381ba None None] MFA Rules not processed for user `5509519254574e50a70c84ae895acd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:45:36.580210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dcb037dd-c249-414d-94ad-403bcc0381ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:36.596900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:45:36.635590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1144/2293] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:45:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:45:43.494465 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede7a7-1a62-4a43-9cb2-cb0786fe2496 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:43.494959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede7a7-1a62-4a43-9cb2-cb0786fe2496 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:43.495200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede7a7-1a62-4a43-9cb2-cb0786fe2496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:43.517438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-daede7a7-1a62-4a43-9cb2-cb0786fe2496 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:45:43.518679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-daede7a7-1a62-4a43-9cb2-cb0786fe2496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:43.571989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1150/2294] 10.222.0.64 () {58 vars in 967 bytes} [Thu Aug 4 07:45:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:45:43.594672 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb037dd-c249-414d-94ad-403bcc0381ba None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:45:43.596140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dcb037dd-c249-414d-94ad-403bcc0381ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:43.664839 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb037dd-c249-414d-94ad-403bcc0381ba None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:45:43.669803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17ba4-baed-4e00-aeb1-c41a7869ea91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:45:43.670418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17ba4-baed-4e00-aeb1-c41a7869ea91 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:43.670705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17ba4-baed-4e00-aeb1-c41a7869ea91 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:43.670985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17ba4-baed-4e00-aeb1-c41a7869ea91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:43.679804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d17ba4-baed-4e00-aeb1-c41a7869ea91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:45:43.681459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d17ba4-baed-4e00-aeb1-c41a7869ea91 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:45:43.690495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1145/2295] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:45:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:45:53.565441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:45:53.686346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=47 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:45:58.856110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94fbac-0b57-49d9-9ccf-2ab9f8fc9ac5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:45:58.856618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94fbac-0b57-49d9-9ccf-2ab9f8fc9ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:45:58.856736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94fbac-0b57-49d9-9ccf-2ab9f8fc9ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:45:58.876259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fd94fbac-0b57-49d9-9ccf-2ab9f8fc9ac5 None None] MFA Rules not processed for user `88d78d9c0369431d88ad85c960f6a0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:45:58.877476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fd94fbac-0b57-49d9-9ccf-2ab9f8fc9ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:45:58.929642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1151/2296] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:45:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:46:08.923873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=25 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:46:21.131285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7482e3-34bc-4575-a3fc-20197d367350 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:46:21.133500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7482e3-34bc-4575-a3fc-20197d367350 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:46:21.133500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7482e3-34bc-4575-a3fc-20197d367350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:46:21.150563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2f7482e3-34bc-4575-a3fc-20197d367350 None None] MFA Rules not processed for user `e44a63323cfb40a2bb7a5e83d8636f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:46:21.151773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2f7482e3-34bc-4575-a3fc-20197d367350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:46:21.206214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1146/2297] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:46:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:46:31.199361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=25 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:46:35.024347 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb75399-43b5-4777-a34a-52406a4cfeca None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:46:35.024867 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb75399-43b5-4777-a34a-52406a4cfeca None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:46:35.024867 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb75399-43b5-4777-a34a-52406a4cfeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:46:35.044003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-aeb75399-43b5-4777-a34a-52406a4cfeca None None] MFA Rules not processed for user `e5fe46987d5942b3a0ec5495aa0040ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:46:35.045294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aeb75399-43b5-4777-a34a-52406a4cfeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:46:35.099713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1152/2298] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:46:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:46:39.100131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7482e3-34bc-4575-a3fc-20197d367350 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:46:39.101562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2f7482e3-34bc-4575-a3fc-20197d367350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:46:39.208501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7482e3-34bc-4575-a3fc-20197d367350 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:46:39.212420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b8458-d421-4d25-9cff-8b1f5d028e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:46:39.213288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b8458-d421-4d25-9cff-8b1f5d028e8c service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:46:39.213954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b8458-d421-4d25-9cff-8b1f5d028e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:46:39.214410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b8458-d421-4d25-9cff-8b1f5d028e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:46:39.216607 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-db1b8458-d421-4d25-9cff-8b1f5d028e8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:46:39.306429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b8458-d421-4d25-9cff-8b1f5d028e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:46:39.308126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b8458-d421-4d25-9cff-8b1f5d028e8c service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:46:39.321870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1147/2299] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:46:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:46:42.581245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb75399-43b5-4777-a34a-52406a4cfeca None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:46:42.583295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aeb75399-43b5-4777-a34a-52406a4cfeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:46:42.638284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb75399-43b5-4777-a34a-52406a4cfeca None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:46:42.641838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7f3ca-aea1-4900-94d5-d717edca2cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:46:42.642445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7f3ca-aea1-4900-94d5-d717edca2cbe service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:46:42.642727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7f3ca-aea1-4900-94d5-d717edca2cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:46:42.643002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7f3ca-aea1-4900-94d5-d717edca2cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:46:42.645437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7df7f3ca-aea1-4900-94d5-d717edca2cbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:46:42.658394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7f3ca-aea1-4900-94d5-d717edca2cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:46:42.659826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7f3ca-aea1-4900-94d5-d717edca2cbe service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:46:42.670822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1153/2300] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:46:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:46:49.189926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c582ad6-964f-4c3f-abef-790a67cd9a60 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:46:49.190400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c582ad6-964f-4c3f-abef-790a67cd9a60 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:46:49.190526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c582ad6-964f-4c3f-abef-790a67cd9a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:46:49.209628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7c582ad6-964f-4c3f-abef-790a67cd9a60 None None] MFA Rules not processed for user `5509519254574e50a70c84ae895acd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:46:49.210853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7c582ad6-964f-4c3f-abef-790a67cd9a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:46:49.217592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1148/2301] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:46:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:46:50.965157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7f3ca-aea1-4900-94d5-d717edca2cbe service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:46:50.966127 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7df7f3ca-aea1-4900-94d5-d717edca2cbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:46:51.057948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7f3ca-aea1-4900-94d5-d717edca2cbe service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:46:51.078791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824f232-dac7-4028-a747-457cf418d6b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:46:51.079433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824f232-dac7-4028-a747-457cf418d6b6 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:46:51.080324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824f232-dac7-4028-a747-457cf418d6b6 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:46:51.080324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824f232-dac7-4028-a747-457cf418d6b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:46:51.083448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4824f232-dac7-4028-a747-457cf418d6b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:46:51.095835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824f232-dac7-4028-a747-457cf418d6b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:46:51.098429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824f232-dac7-4028-a747-457cf418d6b6 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:46:51.112228 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1154/2302] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:46:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:46:57.052507 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c582ad6-964f-4c3f-abef-790a67cd9a60 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:46:57.062144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c582ad6-964f-4c3f-abef-790a67cd9a60 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:46:57.067439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea24da-f61e-4b15-97c2-b1b1e9d423c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:46:57.068139 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea24da-f61e-4b15-97c2-b1b1e9d423c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:46:57.068423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea24da-f61e-4b15-97c2-b1b1e9d423c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:46:57.068701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea24da-f61e-4b15-97c2-b1b1e9d423c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:46:57.070459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f8ea24da-f61e-4b15-97c2-b1b1e9d423c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:46:57.081680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea24da-f61e-4b15-97c2-b1b1e9d423c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:46:57.083678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea24da-f61e-4b15-97c2-b1b1e9d423c9 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:46:57.092629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1149/2303] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:46:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:47:00.295393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824f232-dac7-4028-a747-457cf418d6b6 service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:00.296589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4824f232-dac7-4028-a747-457cf418d6b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:00.305911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:00.316555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824f232-dac7-4028-a747-457cf418d6b6 service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:00.322619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d18ae-063f-4620-9673-ca68bf0ccac6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:00.322967 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d18ae-063f-4620-9673-ca68bf0ccac6 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:00.323249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d18ae-063f-4620-9673-ca68bf0ccac6 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:00.323522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d18ae-063f-4620-9673-ca68bf0ccac6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:00.325365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7b0d18ae-063f-4620-9673-ca68bf0ccac6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:00.337376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d18ae-063f-4620-9673-ca68bf0ccac6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:00.338983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d18ae-063f-4620-9673-ca68bf0ccac6 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:00.349742 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1155/2304] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:01.451123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea24da-f61e-4b15-97c2-b1b1e9d423c9 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:01.452846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f8ea24da-f61e-4b15-97c2-b1b1e9d423c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:01.461396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=64 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:01.466593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea24da-f61e-4b15-97c2-b1b1e9d423c9 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:01.470177 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8761e81-cb3a-4681-ac9b-325c74dd154b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:01.470784 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8761e81-cb3a-4681-ac9b-325c74dd154b service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:01.471086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8761e81-cb3a-4681-ac9b-325c74dd154b service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:01.471373 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8761e81-cb3a-4681-ac9b-325c74dd154b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:01.473857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b8761e81-cb3a-4681-ac9b-325c74dd154b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:01.483133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8761e81-cb3a-4681-ac9b-325c74dd154b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:01.484570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8761e81-cb3a-4681-ac9b-325c74dd154b service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:01.493068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1150/2305] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:47:08.338023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d18ae-063f-4620-9673-ca68bf0ccac6 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:08.348687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d18ae-063f-4620-9673-ca68bf0ccac6 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:08.352420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd43bf-58a3-4296-9cc5-86046e90ce35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:08.353099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd43bf-58a3-4296-9cc5-86046e90ce35 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:08.353444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd43bf-58a3-4296-9cc5-86046e90ce35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:08.353783 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd43bf-58a3-4296-9cc5-86046e90ce35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:08.355599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0dbd43bf-58a3-4296-9cc5-86046e90ce35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:08.424589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd43bf-58a3-4296-9cc5-86046e90ce35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:08.426680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd43bf-58a3-4296-9cc5-86046e90ce35 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:08.436978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1156/2306] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:11.593147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:11.933664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8761e81-cb3a-4681-ac9b-325c74dd154b service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:11.935149 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b8761e81-cb3a-4681-ac9b-325c74dd154b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:11.951224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8761e81-cb3a-4681-ac9b-325c74dd154b service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:11.957721 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a9efb-0512-405e-9a84-7f4157b0664e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:11.958279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a9efb-0512-405e-9a84-7f4157b0664e service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:11.958666 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a9efb-0512-405e-9a84-7f4157b0664e service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:11.959103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a9efb-0512-405e-9a84-7f4157b0664e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:11.961132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f56a9efb-0512-405e-9a84-7f4157b0664e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:11.975215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a9efb-0512-405e-9a84-7f4157b0664e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:11.976954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a9efb-0512-405e-9a84-7f4157b0664e service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:11.987412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1151/2307] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:12.066319 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd43bf-58a3-4296-9cc5-86046e90ce35 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:12.067945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0dbd43bf-58a3-4296-9cc5-86046e90ce35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:12.081480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd43bf-58a3-4296-9cc5-86046e90ce35 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:12.085914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140351d-d5a0-4d67-bbb6-5e9b7f5e9435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:12.086580 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140351d-d5a0-4d67-bbb6-5e9b7f5e9435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:12.086864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140351d-d5a0-4d67-bbb6-5e9b7f5e9435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:12.087141 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140351d-d5a0-4d67-bbb6-5e9b7f5e9435 admin admin] PATH_INFO: `/v3/users/88d78d9c0369431d88ad85c960f6a0ba` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:12.088327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140351d-d5a0-4d67-bbb6-5e9b7f5e9435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d78d9c0369431d88ad85c960f6a0ba)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:12.090246 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140351d-d5a0-4d67-bbb6-5e9b7f5e9435 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:12.287695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e140351d-d5a0-4d67-bbb6-5e9b7f5e9435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d78d9c0369431d88ad85c960f6a0ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:12.288291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e140351d-d5a0-4d67-bbb6-5e9b7f5e9435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d78d9c0369431d88ad85c960f6a0ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:12.295055 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e140351d-d5a0-4d67-bbb6-5e9b7f5e9435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d78d9c0369431d88ad85c960f6a0ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:12.306590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e140351d-d5a0-4d67-bbb6-5e9b7f5e9435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d78d9c0369431d88ad85c960f6a0ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:12.376134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e140351d-d5a0-4d67-bbb6-5e9b7f5e9435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d78d9c0369431d88ad85c960f6a0ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:12.382896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1157/2308] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:47:12 2022] DELETE /identity/v3/users/88d78d9c0369431d88ad85c960f6a0ba => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:12.669594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a9efb-0512-405e-9a84-7f4157b0664e service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:12.679306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a9efb-0512-405e-9a84-7f4157b0664e service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:12.736085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:12.736735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:12.737048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:12.737329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] PATH_INFO: `/v3/projects/f035e76db134496bbc7184d34d7f26d4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:12.738771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f035e76db134496bbc7184d34d7f26d4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:12.740818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:12.871519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f035e76db134496bbc7184d34d7f26d4', 'request_id': 'req-6319f5a4-ca3b-4687-a429-e7d10c039de0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:12.877777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f035e76db134496bbc7184d34d7f26d4', 'request_id': 'req-6319f5a4-ca3b-4687-a429-e7d10c039de0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:12.970829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f035e76db134496bbc7184d34d7f26d4', 'request_id': 'req-6319f5a4-ca3b-4687-a429-e7d10c039de0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:12.981993 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] The token cache is being invalidate because project f035e76db134496bbc7184d34d7f26d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:47:12.982270 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:12.982895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6319f5a4-ca3b-4687-a429-e7d10c039de0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:12.984583 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1152/2309] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:47:12 2022] DELETE /identity/v3/projects/f035e76db134496bbc7184d34d7f26d4 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:12.990165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b40a2f-0657-4b80-a229-a585ead32e0a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:12.990459 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b40a2f-0657-4b80-a229-a585ead32e0a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:12.990735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b40a2f-0657-4b80-a229-a585ead32e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:13.010411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-56b40a2f-0657-4b80-a229-a585ead32e0a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:13.011594 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-56b40a2f-0657-4b80-a229-a585ead32e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:13.059601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:13.072121 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1158/2310] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:13.080195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ebab4-a99a-425e-96c5-c2620ea25710 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:13.080497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ebab4-a99a-425e-96c5-c2620ea25710 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:13.080760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ebab4-a99a-425e-96c5-c2620ea25710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:13.100106 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9c3ebab4-a99a-425e-96c5-c2620ea25710 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:13.101303 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9c3ebab4-a99a-425e-96c5-c2620ea25710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:13.109441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1153/2311] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:47:13.114304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b40a2f-0657-4b80-a229-a585ead32e0a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:13.123862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b40a2f-0657-4b80-a229-a585ead32e0a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:13.129591 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07afbb4-7a0c-4e63-b95a-5ac1eab98aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:13.130239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07afbb4-7a0c-4e63-b95a-5ac1eab98aef admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:13.130522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07afbb4-7a0c-4e63-b95a-5ac1eab98aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:13.130795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07afbb4-7a0c-4e63-b95a-5ac1eab98aef admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:13.131510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07afbb4-7a0c-4e63-b95a-5ac1eab98aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:47:13.131812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07afbb4-7a0c-4e63-b95a-5ac1eab98aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:13.133430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07afbb4-7a0c-4e63-b95a-5ac1eab98aef admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:13.145489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1159/2312] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:47:13.149994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ebab4-a99a-425e-96c5-c2620ea25710 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:13.159344 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ebab4-a99a-425e-96c5-c2620ea25710 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:13.164779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb447ee-e689-42ef-b5ba-8ac5d63eba57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:13.165389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb447ee-e689-42ef-b5ba-8ac5d63eba57 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:13.165670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb447ee-e689-42ef-b5ba-8ac5d63eba57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:13.165949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb447ee-e689-42ef-b5ba-8ac5d63eba57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:13.166614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb447ee-e689-42ef-b5ba-8ac5d63eba57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:13.168559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb447ee-e689-42ef-b5ba-8ac5d63eba57 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:13.251886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1154/2313] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:47:13 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:13.256160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07afbb4-7a0c-4e63-b95a-5ac1eab98aef admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:13.266849 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07afbb4-7a0c-4e63-b95a-5ac1eab98aef admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:13.324483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0c7c4-5135-45f6-8b51-5adad0ef8700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:13.325093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0c7c4-5135-45f6-8b51-5adad0ef8700 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:13.325379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0c7c4-5135-45f6-8b51-5adad0ef8700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:13.325655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0c7c4-5135-45f6-8b51-5adad0ef8700 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:13.326320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0c7c4-5135-45f6-8b51-5adad0ef8700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:13.328288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0c7c4-5135-45f6-8b51-5adad0ef8700 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:13.404980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1160/2314] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:47:13 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:13.408980 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb447ee-e689-42ef-b5ba-8ac5d63eba57 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:13.421126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb447ee-e689-42ef-b5ba-8ac5d63eba57 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:13.426384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e2006-5009-486d-af1b-95d5529d01eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:13.426999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e2006-5009-486d-af1b-95d5529d01eb admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:13.427284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e2006-5009-486d-af1b-95d5529d01eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:13.427566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e2006-5009-486d-af1b-95d5529d01eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:13.428297 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e2006-5009-486d-af1b-95d5529d01eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:13.429899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e2006-5009-486d-af1b-95d5529d01eb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:13.439772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1155/2315] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:13.443687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0c7c4-5135-45f6-8b51-5adad0ef8700 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:13.453859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0c7c4-5135-45f6-8b51-5adad0ef8700 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:13.458552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68207c-450b-4266-bb50-80f91d4aa2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:13.459149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68207c-450b-4266-bb50-80f91d4aa2ff admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:13.459433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68207c-450b-4266-bb50-80f91d4aa2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:13.459711 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68207c-450b-4266-bb50-80f91d4aa2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:13.460442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68207c-450b-4266-bb50-80f91d4aa2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:13.462053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68207c-450b-4266-bb50-80f91d4aa2ff admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:13.473441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1161/2316] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:47:13.477800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e2006-5009-486d-af1b-95d5529d01eb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:13.487765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e2006-5009-486d-af1b-95d5529d01eb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:13.493013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e85bd-f3f2-451e-9b76-20ab000a709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:13.493700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e85bd-f3f2-451e-9b76-20ab000a709a admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:13.493986 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e85bd-f3f2-451e-9b76-20ab000a709a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:13.494267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e85bd-f3f2-451e-9b76-20ab000a709a admin admin] PATH_INFO: `/v3/projects/72ee8449011a466bbae45df7d6b2c603/users/388089b05c074d7ba556b7600cc1bece/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:13.509825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e85bd-f3f2-451e-9b76-20ab000a709a admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ee8449011a466bbae45df7d6b2c603, user_id=388089b05c074d7ba556b7600cc1bece, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:13.512538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e85bd-f3f2-451e-9b76-20ab000a709a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:13.598402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1156/2317] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:13 2022] PUT /identity/v3/projects/72ee8449011a466bbae45df7d6b2c603/users/388089b05c074d7ba556b7600cc1bece/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:47:13.608994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966350c4-98b2-4047-a37e-466eff2f0ca3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:13.609363 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966350c4-98b2-4047-a37e-466eff2f0ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:13.609641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966350c4-98b2-4047-a37e-466eff2f0ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:13.638099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e85bd-f3f2-451e-9b76-20ab000a709a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:13.639309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-966350c4-98b2-4047-a37e-466eff2f0ca3 None None] MFA Rules not processed for user `388089b05c074d7ba556b7600cc1bece`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:13.639879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-354e85bd-f3f2-451e-9b76-20ab000a709a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:13.640514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-966350c4-98b2-4047-a37e-466eff2f0ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:13.654818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:13.708394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1162/2318] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:13.716191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e85bd-f3f2-451e-9b76-20ab000a709a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:13.719908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966350c4-98b2-4047-a37e-466eff2f0ca3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:13.721295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-966350c4-98b2-4047-a37e-466eff2f0ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:13.722060 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ca3ad-92f9-4054-a709-cde572aca639 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:13.722727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ca3ad-92f9-4054-a709-cde572aca639 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:13.723220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ca3ad-92f9-4054-a709-cde572aca639 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:13.723789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ca3ad-92f9-4054-a709-cde572aca639 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:13.725596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-605ca3ad-92f9-4054-a709-cde572aca639 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:13.737941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966350c4-98b2-4047-a37e-466eff2f0ca3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:13.744039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b1d87-c900-427d-81f7-ff74d82a473c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:13.744660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b1d87-c900-427d-81f7-ff74d82a473c service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:13.744965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b1d87-c900-427d-81f7-ff74d82a473c service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:13.748481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b1d87-c900-427d-81f7-ff74d82a473c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:13.753383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b1d87-c900-427d-81f7-ff74d82a473c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:13.755692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b1d87-c900-427d-81f7-ff74d82a473c service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:13.764855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1163/2319] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:47:13.789702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ca3ad-92f9-4054-a709-cde572aca639 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:13.791349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ca3ad-92f9-4054-a709-cde572aca639 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:13.802368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1157/2320] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:17.085487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b1d87-c900-427d-81f7-ff74d82a473c service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:17.124426 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:17.146062 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b1d87-c900-427d-81f7-ff74d82a473c service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:17.150641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:17.152728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:17.152728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:17.152728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] PATH_INFO: `/v3/users/388089b05c074d7ba556b7600cc1bece` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:17.153024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=388089b05c074d7ba556b7600cc1bece)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:17.154953 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:17.338560 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388089b05c074d7ba556b7600cc1bece'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:17.339127 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388089b05c074d7ba556b7600cc1bece'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:17.346254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388089b05c074d7ba556b7600cc1bece'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:17.358748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388089b05c074d7ba556b7600cc1bece'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:17.459773 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388089b05c074d7ba556b7600cc1bece'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:17.466128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1164/2321] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:47:17 2022] DELETE /identity/v3/users/388089b05c074d7ba556b7600cc1bece => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:17.470637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5cb65-9c24-48f9-a2cd-88cc8ac9f249 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:17.471089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5cb65-9c24-48f9-a2cd-88cc8ac9f249 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:17.471203 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5cb65-9c24-48f9-a2cd-88cc8ac9f249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:17.490925 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f2f5cb65-9c24-48f9-a2cd-88cc8ac9f249 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:17.492164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f2f5cb65-9c24-48f9-a2cd-88cc8ac9f249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:17.551560 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1158/2322] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:17.562041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:17.563498 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:17.623964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2a043-373a-4d9d-b072-5066ac97e610 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:17.627740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e716781-1877-4ee5-8c39-9b485edb27f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:17.628648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e716781-1877-4ee5-8c39-9b485edb27f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:17.635126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e716781-1877-4ee5-8c39-9b485edb27f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:17.635126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e716781-1877-4ee5-8c39-9b485edb27f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:17.639008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e716781-1877-4ee5-8c39-9b485edb27f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:17.640533 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e716781-1877-4ee5-8c39-9b485edb27f8 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:17.650072 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1165/2323] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:18.109120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5cb65-9c24-48f9-a2cd-88cc8ac9f249 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:18.110547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f2f5cb65-9c24-48f9-a2cd-88cc8ac9f249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:18.122241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5cb65-9c24-48f9-a2cd-88cc8ac9f249 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:18.126830 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:18.127418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:18.127702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:18.128000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] PATH_INFO: `/v3/projects/72ee8449011a466bbae45df7d6b2c603` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:18.129161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ee8449011a466bbae45df7d6b2c603)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:18.131060 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:18.248281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:18.251169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ee8449011a466bbae45df7d6b2c603', 'request_id': 'req-f62b5774-dd23-4379-9cb0-d529bd332e44'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:18.257421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ee8449011a466bbae45df7d6b2c603', 'request_id': 'req-f62b5774-dd23-4379-9cb0-d529bd332e44'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:18.351320 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ee8449011a466bbae45df7d6b2c603', 'request_id': 'req-f62b5774-dd23-4379-9cb0-d529bd332e44'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:18.361817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] The token cache is being invalidate because project 72ee8449011a466bbae45df7d6b2c603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:47:18.362108 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:18.362772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f62b5774-dd23-4379-9cb0-d529bd332e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:18.364355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1159/2324] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:47:18 2022] DELETE /identity/v3/projects/72ee8449011a466bbae45df7d6b2c603 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:18.369316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609f8aa-2613-47bc-babd-4b30241a6358 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:18.369794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609f8aa-2613-47bc-babd-4b30241a6358 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:18.369892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609f8aa-2613-47bc-babd-4b30241a6358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:18.389474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9609f8aa-2613-47bc-babd-4b30241a6358 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:18.390667 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9609f8aa-2613-47bc-babd-4b30241a6358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:18.444821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:18.453052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1166/2325] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:18.465577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cadbf6-5c75-40af-a9bf-bc267773c0c5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:18.465878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cadbf6-5c75-40af-a9bf-bc267773c0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:18.466167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cadbf6-5c75-40af-a9bf-bc267773c0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:18.485933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-13cadbf6-5c75-40af-a9bf-bc267773c0c5 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:18.487132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-13cadbf6-5c75-40af-a9bf-bc267773c0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:18.495924 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1160/2326] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:18.500130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609f8aa-2613-47bc-babd-4b30241a6358 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:18.510356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609f8aa-2613-47bc-babd-4b30241a6358 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:18.515489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b340fe-bb76-4d68-b047-57dd3fdd4516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:18.516132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b340fe-bb76-4d68-b047-57dd3fdd4516 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:18.516419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b340fe-bb76-4d68-b047-57dd3fdd4516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:18.516695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b340fe-bb76-4d68-b047-57dd3fdd4516 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:18.517388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b340fe-bb76-4d68-b047-57dd3fdd4516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:47:18.517696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b340fe-bb76-4d68-b047-57dd3fdd4516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:18.520091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b340fe-bb76-4d68-b047-57dd3fdd4516 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:18.531526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1167/2327] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:47:18.536258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cadbf6-5c75-40af-a9bf-bc267773c0c5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:18.545409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cadbf6-5c75-40af-a9bf-bc267773c0c5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:18.549866 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93225f-fdd7-4e49-a5c2-012989553470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:18.550483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93225f-fdd7-4e49-a5c2-012989553470 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:18.550768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93225f-fdd7-4e49-a5c2-012989553470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:18.551045 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93225f-fdd7-4e49-a5c2-012989553470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:18.551702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93225f-fdd7-4e49-a5c2-012989553470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:18.553633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93225f-fdd7-4e49-a5c2-012989553470 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:18.651147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1161/2328] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:47:18 2022] POST /identity/v3/projects => generated 405 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:18.655593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b340fe-bb76-4d68-b047-57dd3fdd4516 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:18.666289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b340fe-bb76-4d68-b047-57dd3fdd4516 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:18.721397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca057c2-0b1b-4dde-90c5-077dfcedbf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:18.722005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca057c2-0b1b-4dde-90c5-077dfcedbf91 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:18.722324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca057c2-0b1b-4dde-90c5-077dfcedbf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:18.722607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca057c2-0b1b-4dde-90c5-077dfcedbf91 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:18.723266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca057c2-0b1b-4dde-90c5-077dfcedbf91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:18.725210 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca057c2-0b1b-4dde-90c5-077dfcedbf91 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:18.848568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1168/2329] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:47:18 2022] POST /identity/v3/users => generated 303 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:18.852674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93225f-fdd7-4e49-a5c2-012989553470 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:18.862842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93225f-fdd7-4e49-a5c2-012989553470 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:18.867925 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bddd85-0aac-4386-9fa5-640120f0736b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:18.868554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bddd85-0aac-4386-9fa5-640120f0736b admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:18.868835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bddd85-0aac-4386-9fa5-640120f0736b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:18.869111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bddd85-0aac-4386-9fa5-640120f0736b admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:18.869856 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bddd85-0aac-4386-9fa5-640120f0736b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:18.871436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bddd85-0aac-4386-9fa5-640120f0736b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:18.882009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1162/2330] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:18.885838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca057c2-0b1b-4dde-90c5-077dfcedbf91 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:18.896164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca057c2-0b1b-4dde-90c5-077dfcedbf91 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:18.900937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93e9ec-6e9c-4fab-89c6-1c64b28888a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:18.901541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93e9ec-6e9c-4fab-89c6-1c64b28888a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:18.901825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93e9ec-6e9c-4fab-89c6-1c64b28888a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:18.902099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93e9ec-6e9c-4fab-89c6-1c64b28888a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:18.902850 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93e9ec-6e9c-4fab-89c6-1c64b28888a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:18.904520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93e9ec-6e9c-4fab-89c6-1c64b28888a3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:18.915446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1169/2331] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:18.919413 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bddd85-0aac-4386-9fa5-640120f0736b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:18.929622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bddd85-0aac-4386-9fa5-640120f0736b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:18.934131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ba408-8cf1-4f30-b718-5ec7630e82c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:18.934781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ba408-8cf1-4f30-b718-5ec7630e82c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:18.935060 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ba408-8cf1-4f30-b718-5ec7630e82c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:18.935335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ba408-8cf1-4f30-b718-5ec7630e82c1 admin admin] PATH_INFO: `/v3/projects/e074345aaca8407091468c2638f2ac23/users/acfde3dd16e2417aae35f5335f911aad/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:18.950257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ba408-8cf1-4f30-b718-5ec7630e82c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e074345aaca8407091468c2638f2ac23, user_id=acfde3dd16e2417aae35f5335f911aad, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:18.952960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ba408-8cf1-4f30-b718-5ec7630e82c1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:19.068520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1163/2332] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:18 2022] PUT /identity/v3/projects/e074345aaca8407091468c2638f2ac23/users/acfde3dd16e2417aae35f5335f911aad/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:19.079079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf40afd-9ed2-40fd-8466-1200172f9760 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:19.079389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf40afd-9ed2-40fd-8466-1200172f9760 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:19.079667 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf40afd-9ed2-40fd-8466-1200172f9760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:19.107282 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-eaf40afd-9ed2-40fd-8466-1200172f9760 None None] MFA Rules not processed for user `acfde3dd16e2417aae35f5335f911aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:19.108527 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eaf40afd-9ed2-40fd-8466-1200172f9760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:19.175306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1170/2333] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:19 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:19.180646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ba408-8cf1-4f30-b718-5ec7630e82c1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:19.252679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ba408-8cf1-4f30-b718-5ec7630e82c1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:19.257338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439328e-83ae-4fdf-b46b-726f7f3d5725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:19.257943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439328e-83ae-4fdf-b46b-726f7f3d5725 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:19.258224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439328e-83ae-4fdf-b46b-726f7f3d5725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:19.258496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439328e-83ae-4fdf-b46b-726f7f3d5725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:19.259150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439328e-83ae-4fdf-b46b-726f7f3d5725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:19.261098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439328e-83ae-4fdf-b46b-726f7f3d5725 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:19.367097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1164/2334] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:47:19 2022] POST /identity/v3/projects => generated 405 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:19.371148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf40afd-9ed2-40fd-8466-1200172f9760 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:19.381999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf40afd-9ed2-40fd-8466-1200172f9760 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:19.430588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:19.437307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045e2ee-b5f7-4769-bf38-0c4e86ef09f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:19.437915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045e2ee-b5f7-4769-bf38-0c4e86ef09f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:19.438192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045e2ee-b5f7-4769-bf38-0c4e86ef09f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:19.438461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045e2ee-b5f7-4769-bf38-0c4e86ef09f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:19.439104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045e2ee-b5f7-4769-bf38-0c4e86ef09f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:19.441380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045e2ee-b5f7-4769-bf38-0c4e86ef09f0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:19.593990 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1171/2335] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:47:19 2022] POST /identity/v3/users => generated 309 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:19.598678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439328e-83ae-4fdf-b46b-726f7f3d5725 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:19.608758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439328e-83ae-4fdf-b46b-726f7f3d5725 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:19.613853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5cf3ad-317e-46a9-af37-9856c14d9da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:19.614468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cf3ad-317e-46a9-af37-9856c14d9da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:19.614738 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cf3ad-317e-46a9-af37-9856c14d9da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:19.615012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cf3ad-317e-46a9-af37-9856c14d9da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:19.615707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5cf3ad-317e-46a9-af37-9856c14d9da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:19.617334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5cf3ad-317e-46a9-af37-9856c14d9da5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:19.627698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1165/2336] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:19.632469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045e2ee-b5f7-4769-bf38-0c4e86ef09f0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:19.642609 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045e2ee-b5f7-4769-bf38-0c4e86ef09f0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:19.647254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630162-18c5-4db4-ade0-f7c0a16b249d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:19.647910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630162-18c5-4db4-ade0-f7c0a16b249d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:19.654000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630162-18c5-4db4-ade0-f7c0a16b249d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:19.654000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630162-18c5-4db4-ade0-f7c0a16b249d admin admin] PATH_INFO: `/v3/projects/1ba0ab6e3e6948aba2c481e0abb9be20/users/fdba1ce956684edb92139b98f02adea6/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:19.662818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0630162-18c5-4db4-ade0-f7c0a16b249d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba0ab6e3e6948aba2c481e0abb9be20, user_id=fdba1ce956684edb92139b98f02adea6, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:19.665508 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0630162-18c5-4db4-ade0-f7c0a16b249d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:19.817096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1172/2337] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:19 2022] PUT /identity/v3/projects/1ba0ab6e3e6948aba2c481e0abb9be20/users/fdba1ce956684edb92139b98f02adea6/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:19.821459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5cf3ad-317e-46a9-af37-9856c14d9da5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:19.831669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5cf3ad-317e-46a9-af37-9856c14d9da5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:19.871913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:19.886782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bcd72-822e-4da3-8b47-b76d2b69a9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:19.887403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bcd72-822e-4da3-8b47-b76d2b69a9ac admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:19.887688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bcd72-822e-4da3-8b47-b76d2b69a9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:19.887967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bcd72-822e-4da3-8b47-b76d2b69a9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:19.888684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bcd72-822e-4da3-8b47-b76d2b69a9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:19.890337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bcd72-822e-4da3-8b47-b76d2b69a9ac admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:19.900076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1166/2338] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:19.904469 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630162-18c5-4db4-ade0-f7c0a16b249d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:19.915342 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630162-18c5-4db4-ade0-f7c0a16b249d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:19.919921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7dc98-3c8f-4a07-b605-4060dbc2b9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:19.920633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7dc98-3c8f-4a07-b605-4060dbc2b9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:19.920910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7dc98-3c8f-4a07-b605-4060dbc2b9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:19.921181 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7dc98-3c8f-4a07-b605-4060dbc2b9b1 admin admin] PATH_INFO: `/v3/projects/1ba0ab6e3e6948aba2c481e0abb9be20/users/fdba1ce956684edb92139b98f02adea6/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:19.923108 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7dc98-3c8f-4a07-b605-4060dbc2b9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba0ab6e3e6948aba2c481e0abb9be20, user_id=fdba1ce956684edb92139b98f02adea6, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:19.926436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7dc98-3c8f-4a07-b605-4060dbc2b9b1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:19.936467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1173/2339] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:19 2022] PUT /identity/v3/projects/1ba0ab6e3e6948aba2c481e0abb9be20/users/fdba1ce956684edb92139b98f02adea6/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:19.947776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ae5fe-a114-495d-81a4-fbaa906ee35a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:19.952051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ae5fe-a114-495d-81a4-fbaa906ee35a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:19.952051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ae5fe-a114-495d-81a4-fbaa906ee35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:19.978483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ce8ae5fe-a114-495d-81a4-fbaa906ee35a None None] MFA Rules not processed for user `fdba1ce956684edb92139b98f02adea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:19.979698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ce8ae5fe-a114-495d-81a4-fbaa906ee35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:20.047432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1167/2340] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:19 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:20.058548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7dc98-3c8f-4a07-b605-4060dbc2b9b1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:20.060241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b7b7dc98-3c8f-4a07-b605-4060dbc2b9b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:20.120234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7dc98-3c8f-4a07-b605-4060dbc2b9b1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:20.123745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fb6ed-4cff-4b1e-a9f0-8ad038e2a63a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:20.124394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fb6ed-4cff-4b1e-a9f0-8ad038e2a63a service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:20.124674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fb6ed-4cff-4b1e-a9f0-8ad038e2a63a service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:20.124947 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fb6ed-4cff-4b1e-a9f0-8ad038e2a63a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:20.134035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659fb6ed-4cff-4b1e-a9f0-8ad038e2a63a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:20.135461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659fb6ed-4cff-4b1e-a9f0-8ad038e2a63a service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:20.145036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1174/2341] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:20 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:20.296289 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ae5fe-a114-495d-81a4-fbaa906ee35a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:20.358035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ae5fe-a114-495d-81a4-fbaa906ee35a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:20.362398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:20.363087 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:20.363366 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:20.363639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] PATH_INFO: `/v3/users/acfde3dd16e2417aae35f5335f911aad` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:20.364847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acfde3dd16e2417aae35f5335f911aad)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:20.366785 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:20.599163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acfde3dd16e2417aae35f5335f911aad'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:20.599826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acfde3dd16e2417aae35f5335f911aad'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:20.600469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acfde3dd16e2417aae35f5335f911aad'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:20.612495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acfde3dd16e2417aae35f5335f911aad'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:20.615870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:20.713052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acfde3dd16e2417aae35f5335f911aad'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:20.726542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1168/2342] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:47:20 2022] DELETE /identity/v3/users/acfde3dd16e2417aae35f5335f911aad => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:20.731278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df2ad9-f3b5-4ec8-9ac6-d54c97e08f62 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:20.731737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df2ad9-f3b5-4ec8-9ac6-d54c97e08f62 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:20.731831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df2ad9-f3b5-4ec8-9ac6-d54c97e08f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:20.751271 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-33df2ad9-f3b5-4ec8-9ac6-d54c97e08f62 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:20.752484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-33df2ad9-f3b5-4ec8-9ac6-d54c97e08f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:20.809884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1175/2343] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:47:20.819649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:20.821072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:20.876704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89d42d-e0bc-49a1-9d4e-ef74cd264997 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:20.880354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4311f1-824d-44d7-a0a0-732803b5c251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:20.880968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4311f1-824d-44d7-a0a0-732803b5c251 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:20.881252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4311f1-824d-44d7-a0a0-732803b5c251 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:20.881529 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4311f1-824d-44d7-a0a0-732803b5c251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:20.890585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4311f1-824d-44d7-a0a0-732803b5c251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:20.892048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4311f1-824d-44d7-a0a0-732803b5c251 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:20.901433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1169/2344] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:47:21.292949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df2ad9-f3b5-4ec8-9ac6-d54c97e08f62 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:21.294466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-33df2ad9-f3b5-4ec8-9ac6-d54c97e08f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:21.307697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df2ad9-f3b5-4ec8-9ac6-d54c97e08f62 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:21.312561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:21.313197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:21.313598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:21.313837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] PATH_INFO: `/v3/projects/e074345aaca8407091468c2638f2ac23` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:21.315000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e074345aaca8407091468c2638f2ac23)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:21.317873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:21.415348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:21.418174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e074345aaca8407091468c2638f2ac23', 'request_id': 'req-a9d64eeb-2b2f-432f-b053-97232fd284d6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:21.516743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e074345aaca8407091468c2638f2ac23', 'request_id': 'req-a9d64eeb-2b2f-432f-b053-97232fd284d6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:21.522597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e074345aaca8407091468c2638f2ac23', 'request_id': 'req-a9d64eeb-2b2f-432f-b053-97232fd284d6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:21.533747 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] The token cache is being invalidate because project e074345aaca8407091468c2638f2ac23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:47:21.534018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:21.534668 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:21.536998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1176/2345] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:47:21 2022] DELETE /identity/v3/projects/e074345aaca8407091468c2638f2ac23 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:21.541010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4311f1-824d-44d7-a0a0-732803b5c251 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:21.551738 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4311f1-824d-44d7-a0a0-732803b5c251 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:21.612735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd4c23-cf94-4b28-9e64-63b14324f752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:21.614331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd4c23-cf94-4b28-9e64-63b14324f752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:21.614662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd4c23-cf94-4b28-9e64-63b14324f752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:21.615004 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd4c23-cf94-4b28-9e64-63b14324f752 admin admin] PATH_INFO: `/v3/users/fdba1ce956684edb92139b98f02adea6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:21.616191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd4c23-cf94-4b28-9e64-63b14324f752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdba1ce956684edb92139b98f02adea6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:21.618216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd4c23-cf94-4b28-9e64-63b14324f752 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:21.801213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94fd4c23-cf94-4b28-9e64-63b14324f752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdba1ce956684edb92139b98f02adea6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:21.801882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94fd4c23-cf94-4b28-9e64-63b14324f752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdba1ce956684edb92139b98f02adea6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:21.802494 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94fd4c23-cf94-4b28-9e64-63b14324f752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdba1ce956684edb92139b98f02adea6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:21.814256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94fd4c23-cf94-4b28-9e64-63b14324f752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdba1ce956684edb92139b98f02adea6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:21.900076 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94fd4c23-cf94-4b28-9e64-63b14324f752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdba1ce956684edb92139b98f02adea6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:21.913255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1170/2346] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:47:21 2022] DELETE /identity/v3/users/fdba1ce956684edb92139b98f02adea6 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:22.387389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:22.388920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:22.404105 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d64eeb-2b2f-432f-b053-97232fd284d6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:22.462850 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:22.463493 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:22.463778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:22.464118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] PATH_INFO: `/v3/projects/1ba0ab6e3e6948aba2c481e0abb9be20` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:22.465267 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba0ab6e3e6948aba2c481e0abb9be20)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:22.467247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:22.589258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba0ab6e3e6948aba2c481e0abb9be20', 'request_id': 'req-6a425afb-7df4-4a1b-b625-e75708d50162'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:22.660814 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba0ab6e3e6948aba2c481e0abb9be20', 'request_id': 'req-6a425afb-7df4-4a1b-b625-e75708d50162'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:22.667261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba0ab6e3e6948aba2c481e0abb9be20', 'request_id': 'req-6a425afb-7df4-4a1b-b625-e75708d50162'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:22.678277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] The token cache is being invalidate because project 1ba0ab6e3e6948aba2c481e0abb9be20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:47:22.678566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:22.679203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a425afb-7df4-4a1b-b625-e75708d50162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:22.680743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1177/2347] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:47:22 2022] DELETE /identity/v3/projects/1ba0ab6e3e6948aba2c481e0abb9be20 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:22.685767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713dbb7-ddbc-4eb0-81ff-e4614ee7a0bc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:22.686276 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713dbb7-ddbc-4eb0-81ff-e4614ee7a0bc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:22.686365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713dbb7-ddbc-4eb0-81ff-e4614ee7a0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:22.701004 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:22.708355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0713dbb7-ddbc-4eb0-81ff-e4614ee7a0bc None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:22.709582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0713dbb7-ddbc-4eb0-81ff-e4614ee7a0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:22.768180 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1171/2348] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:47:22.776721 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170eded7-c3ad-4ac5-a05e-74a065cb6bdd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:22.777008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170eded7-c3ad-4ac5-a05e-74a065cb6bdd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:22.777286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170eded7-c3ad-4ac5-a05e-74a065cb6bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:22.798015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-170eded7-c3ad-4ac5-a05e-74a065cb6bdd None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:22.799219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-170eded7-c3ad-4ac5-a05e-74a065cb6bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:22.807456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1178/2349] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:22.811915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713dbb7-ddbc-4eb0-81ff-e4614ee7a0bc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:22.821888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713dbb7-ddbc-4eb0-81ff-e4614ee7a0bc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:22.826488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f7647-fd34-491e-8c86-1500107b7089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:22.827232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f7647-fd34-491e-8c86-1500107b7089 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:22.827542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f7647-fd34-491e-8c86-1500107b7089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:22.832282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f7647-fd34-491e-8c86-1500107b7089 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:22.832282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f7647-fd34-491e-8c86-1500107b7089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:47:22.832282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f7647-fd34-491e-8c86-1500107b7089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:22.832282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f7647-fd34-491e-8c86-1500107b7089 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:22.842959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1172/2350] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:47:22.848325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170eded7-c3ad-4ac5-a05e-74a065cb6bdd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:22.857337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170eded7-c3ad-4ac5-a05e-74a065cb6bdd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:22.864392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032f4d6-d6f0-4603-8010-d1747e204fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:22.864392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032f4d6-d6f0-4603-8010-d1747e204fce admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:22.864392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032f4d6-d6f0-4603-8010-d1747e204fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:22.864392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032f4d6-d6f0-4603-8010-d1747e204fce admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:22.864787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032f4d6-d6f0-4603-8010-d1747e204fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:22.865608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032f4d6-d6f0-4603-8010-d1747e204fce admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:22.993939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1179/2351] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:47:22 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:22.997478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f7647-fd34-491e-8c86-1500107b7089 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:23.007960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f7647-fd34-491e-8c86-1500107b7089 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:23.067252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20887422-2619-470f-99ad-ed5e12f80c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:23.067885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887422-2619-470f-99ad-ed5e12f80c2e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:23.068197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887422-2619-470f-99ad-ed5e12f80c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:23.068475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887422-2619-470f-99ad-ed5e12f80c2e admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:23.069135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20887422-2619-470f-99ad-ed5e12f80c2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:23.071124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20887422-2619-470f-99ad-ed5e12f80c2e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:23.182371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1173/2352] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:47:22 2022] POST /identity/v3/users => generated 283 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:23.186483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032f4d6-d6f0-4603-8010-d1747e204fce admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:23.197597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032f4d6-d6f0-4603-8010-d1747e204fce admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:23.202259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58dc42-b192-46cb-99f5-d8097b72d5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:23.202862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58dc42-b192-46cb-99f5-d8097b72d5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:23.203142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58dc42-b192-46cb-99f5-d8097b72d5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:23.203440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58dc42-b192-46cb-99f5-d8097b72d5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:23.204452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58dc42-b192-46cb-99f5-d8097b72d5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:23.206040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58dc42-b192-46cb-99f5-d8097b72d5b0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:23.220985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1180/2353] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:23.222274 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20887422-2619-470f-99ad-ed5e12f80c2e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:23.232760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20887422-2619-470f-99ad-ed5e12f80c2e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:23.237746 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6470f-d247-4a98-8246-20e49b173da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:23.238439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6470f-d247-4a98-8246-20e49b173da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:23.238788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6470f-d247-4a98-8246-20e49b173da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:23.239127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6470f-d247-4a98-8246-20e49b173da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:23.239889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6470f-d247-4a98-8246-20e49b173da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:23.241694 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6470f-d247-4a98-8246-20e49b173da0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:23.252556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1174/2354] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:23.257325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58dc42-b192-46cb-99f5-d8097b72d5b0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:23.267031 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58dc42-b192-46cb-99f5-d8097b72d5b0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:23.274440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e08a18-c182-402a-9875-5df6e7083840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:23.275173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e08a18-c182-402a-9875-5df6e7083840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:23.275460 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e08a18-c182-402a-9875-5df6e7083840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:23.275739 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e08a18-c182-402a-9875-5df6e7083840 admin admin] PATH_INFO: `/v3/projects/5f261aea64af4d3dbd74982361aa5211/users/e48c2c6c397449cc90d8896abc4ae18f/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:23.291422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e08a18-c182-402a-9875-5df6e7083840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f261aea64af4d3dbd74982361aa5211, user_id=e48c2c6c397449cc90d8896abc4ae18f, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:23.294140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e08a18-c182-402a-9875-5df6e7083840 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:23.412442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:23.417274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1181/2355] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:23 2022] PUT /identity/v3/projects/5f261aea64af4d3dbd74982361aa5211/users/e48c2c6c397449cc90d8896abc4ae18f/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:23.428193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3fbf9-bbe2-4c39-ba11-05eaafce6212 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:23.428479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3fbf9-bbe2-4c39-ba11-05eaafce6212 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:23.428752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3fbf9-bbe2-4c39-ba11-05eaafce6212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:23.457297 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6ff3fbf9-bbe2-4c39-ba11-05eaafce6212 None None] MFA Rules not processed for user `e48c2c6c397449cc90d8896abc4ae18f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:23.458591 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6ff3fbf9-bbe2-4c39-ba11-05eaafce6212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:23.528403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1175/2356] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:23.533047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e08a18-c182-402a-9875-5df6e7083840 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:23.592317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e08a18-c182-402a-9875-5df6e7083840 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:23.596944 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5569f4-1cc2-44d2-ad95-3bfae9ff02d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:23.597588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5569f4-1cc2-44d2-ad95-3bfae9ff02d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:23.597858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5569f4-1cc2-44d2-ad95-3bfae9ff02d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:23.598192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5569f4-1cc2-44d2-ad95-3bfae9ff02d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:23.599093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5569f4-1cc2-44d2-ad95-3bfae9ff02d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:23.601347 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5569f4-1cc2-44d2-ad95-3bfae9ff02d7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:23.745193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1182/2357] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:47:23 2022] POST /identity/v3/projects => generated 365 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:47:23.749239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3fbf9-bbe2-4c39-ba11-05eaafce6212 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:23.755238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:23.761392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3fbf9-bbe2-4c39-ba11-05eaafce6212 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:23.817298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59146aae-0d00-435a-ada5-0e40547015eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:23.817924 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146aae-0d00-435a-ada5-0e40547015eb admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:23.818211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146aae-0d00-435a-ada5-0e40547015eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:23.818493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146aae-0d00-435a-ada5-0e40547015eb admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:23.819586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59146aae-0d00-435a-ada5-0e40547015eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:23.821111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59146aae-0d00-435a-ada5-0e40547015eb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:23.922164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1176/2358] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:47:23 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:23.923337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5569f4-1cc2-44d2-ad95-3bfae9ff02d7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:23.934748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5569f4-1cc2-44d2-ad95-3bfae9ff02d7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:23.939392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d4312-fbe2-4304-975b-e81d5fb69882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:23.940025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d4312-fbe2-4304-975b-e81d5fb69882 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:23.940312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d4312-fbe2-4304-975b-e81d5fb69882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:23.940585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d4312-fbe2-4304-975b-e81d5fb69882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:23.941486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d4312-fbe2-4304-975b-e81d5fb69882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:23.943301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d4312-fbe2-4304-975b-e81d5fb69882 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:23.962915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1183/2359] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:23.965263 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59146aae-0d00-435a-ada5-0e40547015eb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:23.976118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59146aae-0d00-435a-ada5-0e40547015eb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:23.980867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08bc19-7dbe-4afe-94ed-d02fdafafadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:23.981538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08bc19-7dbe-4afe-94ed-d02fdafafadd admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:23.981827 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08bc19-7dbe-4afe-94ed-d02fdafafadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:23.982105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08bc19-7dbe-4afe-94ed-d02fdafafadd admin admin] PATH_INFO: `/v3/projects/da39b0637b0042ea8ddf5e8abeb2b64b/users/382cddc26fdf40de8f228a6ed4cfc199/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:23.997642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08bc19-7dbe-4afe-94ed-d02fdafafadd admin admin] RBAC: Authorizing `identity:create_grant(project_id=da39b0637b0042ea8ddf5e8abeb2b64b, user_id=382cddc26fdf40de8f228a6ed4cfc199, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:24.000453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08bc19-7dbe-4afe-94ed-d02fdafafadd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:24.079766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1177/2360] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:23 2022] PUT /identity/v3/projects/da39b0637b0042ea8ddf5e8abeb2b64b/users/382cddc26fdf40de8f228a6ed4cfc199/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:47:24.084496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d4312-fbe2-4304-975b-e81d5fb69882 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:24.094662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d4312-fbe2-4304-975b-e81d5fb69882 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:24.151727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01837bde-6f14-47eb-bac7-50ec649fcf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:24.152381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01837bde-6f14-47eb-bac7-50ec649fcf8f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:24.152664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01837bde-6f14-47eb-bac7-50ec649fcf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:24.152940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01837bde-6f14-47eb-bac7-50ec649fcf8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:24.153632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01837bde-6f14-47eb-bac7-50ec649fcf8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:24.155255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01837bde-6f14-47eb-bac7-50ec649fcf8f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:24.165762 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1184/2361] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:24.170017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08bc19-7dbe-4afe-94ed-d02fdafafadd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:24.180463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08bc19-7dbe-4afe-94ed-d02fdafafadd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:24.185382 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d058fae-55f2-4b0b-96a9-243f99919346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:24.191248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d058fae-55f2-4b0b-96a9-243f99919346 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:24.191248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d058fae-55f2-4b0b-96a9-243f99919346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:24.191248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d058fae-55f2-4b0b-96a9-243f99919346 admin admin] PATH_INFO: `/v3/projects/da39b0637b0042ea8ddf5e8abeb2b64b/users/382cddc26fdf40de8f228a6ed4cfc199/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:24.191248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d058fae-55f2-4b0b-96a9-243f99919346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da39b0637b0042ea8ddf5e8abeb2b64b, user_id=382cddc26fdf40de8f228a6ed4cfc199, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:24.191604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d058fae-55f2-4b0b-96a9-243f99919346 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:24.209941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1178/2362] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:24 2022] PUT /identity/v3/projects/da39b0637b0042ea8ddf5e8abeb2b64b/users/382cddc26fdf40de8f228a6ed4cfc199/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:24.221418 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de0e36-caf0-4e5a-a064-2330041d47fd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:24.221710 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de0e36-caf0-4e5a-a064-2330041d47fd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:24.221992 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de0e36-caf0-4e5a-a064-2330041d47fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:24.250438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-34de0e36-caf0-4e5a-a064-2330041d47fd None None] MFA Rules not processed for user `382cddc26fdf40de8f228a6ed4cfc199`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:24.251571 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-34de0e36-caf0-4e5a-a064-2330041d47fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:24.310275 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:24.319185 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1185/2363] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:24.331292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d058fae-55f2-4b0b-96a9-243f99919346 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:24.334012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6d058fae-55f2-4b0b-96a9-243f99919346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:24.390300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d058fae-55f2-4b0b-96a9-243f99919346 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:24.393918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e6f1e-4a41-4736-aea9-8eccafe8deff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:24.395108 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e6f1e-4a41-4736-aea9-8eccafe8deff service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:24.395481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e6f1e-4a41-4736-aea9-8eccafe8deff service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:24.395789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e6f1e-4a41-4736-aea9-8eccafe8deff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:24.404568 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e6f1e-4a41-4736-aea9-8eccafe8deff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:24.405995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e6f1e-4a41-4736-aea9-8eccafe8deff service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:24.415874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1179/2364] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:25.404894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de0e36-caf0-4e5a-a064-2330041d47fd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:25.466174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de0e36-caf0-4e5a-a064-2330041d47fd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:25.470692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:25.471325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:25.471611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:25.471889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] PATH_INFO: `/v3/users/e48c2c6c397449cc90d8896abc4ae18f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:25.473592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e48c2c6c397449cc90d8896abc4ae18f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:25.475517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:25.680130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e48c2c6c397449cc90d8896abc4ae18f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:25.680787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e48c2c6c397449cc90d8896abc4ae18f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:25.687910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e48c2c6c397449cc90d8896abc4ae18f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:25.704826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e48c2c6c397449cc90d8896abc4ae18f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:25.787879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e48c2c6c397449cc90d8896abc4ae18f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:25.794504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1186/2365] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:47:25 2022] DELETE /identity/v3/users/e48c2c6c397449cc90d8896abc4ae18f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:25.799025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54472318-695a-46eb-8db5-f4dc9a2b34c9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:25.799512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54472318-695a-46eb-8db5-f4dc9a2b34c9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:25.799627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54472318-695a-46eb-8db5-f4dc9a2b34c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:25.818785 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-54472318-695a-46eb-8db5-f4dc9a2b34c9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:25.820021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-54472318-695a-46eb-8db5-f4dc9a2b34c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:25.872846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:25.882260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1180/2366] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:25.891993 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:25.893474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:25.951115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ede2b-9910-42a9-84a1-a641512ac1fa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:25.955149 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c014a-2741-46e9-88be-e6fae18005ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:25.955805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c014a-2741-46e9-88be-e6fae18005ae service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:25.956126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c014a-2741-46e9-88be-e6fae18005ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:25.956409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c014a-2741-46e9-88be-e6fae18005ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:25.966855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c014a-2741-46e9-88be-e6fae18005ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:25.968301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c014a-2741-46e9-88be-e6fae18005ae service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:25.978155 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1187/2367] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:26.448012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54472318-695a-46eb-8db5-f4dc9a2b34c9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:26.449144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-54472318-695a-46eb-8db5-f4dc9a2b34c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:26.462304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54472318-695a-46eb-8db5-f4dc9a2b34c9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:26.467179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:26.467850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:26.468173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:26.468455 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] PATH_INFO: `/v3/projects/5f261aea64af4d3dbd74982361aa5211` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:26.469787 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f261aea64af4d3dbd74982361aa5211)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:26.471779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:26.615565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f261aea64af4d3dbd74982361aa5211', 'request_id': 'req-71a104c1-fbc5-425c-8a37-4407eac1672f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:26.622100 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f261aea64af4d3dbd74982361aa5211', 'request_id': 'req-71a104c1-fbc5-425c-8a37-4407eac1672f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:26.731133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f261aea64af4d3dbd74982361aa5211', 'request_id': 'req-71a104c1-fbc5-425c-8a37-4407eac1672f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:26.746274 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] The token cache is being invalidate because project 5f261aea64af4d3dbd74982361aa5211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:47:26.746274 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:26.746274 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:26.746274 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1181/2368] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:47:26 2022] DELETE /identity/v3/projects/5f261aea64af4d3dbd74982361aa5211 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:26.749730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c014a-2741-46e9-88be-e6fae18005ae service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:26.759741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c014a-2741-46e9-88be-e6fae18005ae service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:26.775351 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:26.821042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37d240-a32b-46f8-b837-828f53c7ccc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:26.821689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37d240-a32b-46f8-b837-828f53c7ccc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:26.822011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37d240-a32b-46f8-b837-828f53c7ccc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:26.826175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37d240-a32b-46f8-b837-828f53c7ccc1 admin admin] PATH_INFO: `/v3/users/382cddc26fdf40de8f228a6ed4cfc199` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:26.826175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37d240-a32b-46f8-b837-828f53c7ccc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=382cddc26fdf40de8f228a6ed4cfc199)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:26.826175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37d240-a32b-46f8-b837-828f53c7ccc1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:27.016328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca37d240-a32b-46f8-b837-828f53c7ccc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382cddc26fdf40de8f228a6ed4cfc199'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:27.016931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca37d240-a32b-46f8-b837-828f53c7ccc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382cddc26fdf40de8f228a6ed4cfc199'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:27.023921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca37d240-a32b-46f8-b837-828f53c7ccc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382cddc26fdf40de8f228a6ed4cfc199'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:27.036106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca37d240-a32b-46f8-b837-828f53c7ccc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382cddc26fdf40de8f228a6ed4cfc199'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:27.125011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca37d240-a32b-46f8-b837-828f53c7ccc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382cddc26fdf40de8f228a6ed4cfc199'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:27.131586 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1188/2369] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:47:26 2022] DELETE /identity/v3/users/382cddc26fdf40de8f228a6ed4cfc199 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:27.578419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:27.578419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:27.590202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a104c1-fbc5-425c-8a37-4407eac1672f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:27.652310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:27.652960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:27.653247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:27.653529 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] PATH_INFO: `/v3/projects/da39b0637b0042ea8ddf5e8abeb2b64b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:27.654700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da39b0637b0042ea8ddf5e8abeb2b64b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:27.656665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:27.838333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da39b0637b0042ea8ddf5e8abeb2b64b', 'request_id': 'req-21df7ea3-ed88-4376-906c-21d5ee2fee74'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:27.844337 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da39b0637b0042ea8ddf5e8abeb2b64b', 'request_id': 'req-21df7ea3-ed88-4376-906c-21d5ee2fee74'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:27.965322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da39b0637b0042ea8ddf5e8abeb2b64b', 'request_id': 'req-21df7ea3-ed88-4376-906c-21d5ee2fee74'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:27.976603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] The token cache is being invalidate because project da39b0637b0042ea8ddf5e8abeb2b64b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:47:27.976897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:27.977523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21df7ea3-ed88-4376-906c-21d5ee2fee74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:27.979070 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1182/2370] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:47:27 2022] DELETE /identity/v3/projects/da39b0637b0042ea8ddf5e8abeb2b64b => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:27.986955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f743d6-f42b-445f-b129-ef1de3f06bb9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:27.986955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f743d6-f42b-445f-b129-ef1de3f06bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:27.986955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f743d6-f42b-445f-b129-ef1de3f06bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:28.004481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-92f743d6-f42b-445f-b129-ef1de3f06bb9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:28.006566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-92f743d6-f42b-445f-b129-ef1de3f06bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:28.066397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1189/2371] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:28.075010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45739522-455d-4162-a1f7-1504bfd48f07 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:28.075307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45739522-455d-4162-a1f7-1504bfd48f07 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:28.075594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45739522-455d-4162-a1f7-1504bfd48f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:28.095636 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-45739522-455d-4162-a1f7-1504bfd48f07 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:28.096801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-45739522-455d-4162-a1f7-1504bfd48f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:28.105270 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1183/2372] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:28.109567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f743d6-f42b-445f-b129-ef1de3f06bb9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:28.119855 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f743d6-f42b-445f-b129-ef1de3f06bb9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:28.124473 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febf797-2e66-4ca8-a2d9-3e5e6d450d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:28.129452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febf797-2e66-4ca8-a2d9-3e5e6d450d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:28.129452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febf797-2e66-4ca8-a2d9-3e5e6d450d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:28.129452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febf797-2e66-4ca8-a2d9-3e5e6d450d2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:28.129452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febf797-2e66-4ca8-a2d9-3e5e6d450d2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:47:28.129452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febf797-2e66-4ca8-a2d9-3e5e6d450d2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:28.129452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febf797-2e66-4ca8-a2d9-3e5e6d450d2c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:28.140575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1190/2373] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:47:28.144962 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45739522-455d-4162-a1f7-1504bfd48f07 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:28.150596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:28.155003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45739522-455d-4162-a1f7-1504bfd48f07 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:28.159729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab6fb9-c0b6-44d9-95cf-2d5e6ccc32f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:28.160377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab6fb9-c0b6-44d9-95cf-2d5e6ccc32f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:28.160685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab6fb9-c0b6-44d9-95cf-2d5e6ccc32f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:28.161560 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab6fb9-c0b6-44d9-95cf-2d5e6ccc32f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:28.162243 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab6fb9-c0b6-44d9-95cf-2d5e6ccc32f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:28.164193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab6fb9-c0b6-44d9-95cf-2d5e6ccc32f2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:28.302454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1184/2374] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:47:28 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:47:28.306699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febf797-2e66-4ca8-a2d9-3e5e6d450d2c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:28.316582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febf797-2e66-4ca8-a2d9-3e5e6d450d2c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:28.338195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:28.376966 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477e0e3-108e-43eb-a375-9267e39d12a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:28.377586 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477e0e3-108e-43eb-a375-9267e39d12a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:28.377870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477e0e3-108e-43eb-a375-9267e39d12a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:28.378310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477e0e3-108e-43eb-a375-9267e39d12a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:28.378810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477e0e3-108e-43eb-a375-9267e39d12a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:28.380772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477e0e3-108e-43eb-a375-9267e39d12a0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:28.506126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1191/2375] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:47:28 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:28.510615 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab6fb9-c0b6-44d9-95cf-2d5e6ccc32f2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:28.521313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab6fb9-c0b6-44d9-95cf-2d5e6ccc32f2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:28.525954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b12c9-2dd9-4fbd-a65f-75caa4259c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:28.526561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b12c9-2dd9-4fbd-a65f-75caa4259c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:28.526846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b12c9-2dd9-4fbd-a65f-75caa4259c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:28.527129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b12c9-2dd9-4fbd-a65f-75caa4259c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:28.527885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b12c9-2dd9-4fbd-a65f-75caa4259c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:28.529513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b12c9-2dd9-4fbd-a65f-75caa4259c57 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:28.540473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1185/2376] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:28.544292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477e0e3-108e-43eb-a375-9267e39d12a0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:28.554587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477e0e3-108e-43eb-a375-9267e39d12a0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:28.559264 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c36ed-f313-4bd2-bd34-3572d6555baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:28.559941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c36ed-f313-4bd2-bd34-3572d6555baa admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:28.560257 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c36ed-f313-4bd2-bd34-3572d6555baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:28.560533 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c36ed-f313-4bd2-bd34-3572d6555baa admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:28.561220 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c36ed-f313-4bd2-bd34-3572d6555baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:28.562823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c36ed-f313-4bd2-bd34-3572d6555baa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:28.572692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1192/2377] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:28.576756 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b12c9-2dd9-4fbd-a65f-75caa4259c57 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:28.587780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b12c9-2dd9-4fbd-a65f-75caa4259c57 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:28.592965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e6850-5632-4ea7-b037-599bf8a0cff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:28.593715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e6850-5632-4ea7-b037-599bf8a0cff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:28.594136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e6850-5632-4ea7-b037-599bf8a0cff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:28.594433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e6850-5632-4ea7-b037-599bf8a0cff9 admin admin] PATH_INFO: `/v3/projects/d458bafdab824cc1adad543dc2da149d/users/17c94d4500f94d4bb68f1379043830f7/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:28.611296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e6850-5632-4ea7-b037-599bf8a0cff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d458bafdab824cc1adad543dc2da149d, user_id=17c94d4500f94d4bb68f1379043830f7, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:28.614562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e6850-5632-4ea7-b037-599bf8a0cff9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:28.771252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1186/2378] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:28 2022] PUT /identity/v3/projects/d458bafdab824cc1adad543dc2da149d/users/17c94d4500f94d4bb68f1379043830f7/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:28.781801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d6cd5-6014-42fa-8dae-78c7eb1323f7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:28.781996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d6cd5-6014-42fa-8dae-78c7eb1323f7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:28.782340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d6cd5-6014-42fa-8dae-78c7eb1323f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:28.811403 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-973d6cd5-6014-42fa-8dae-78c7eb1323f7 None None] MFA Rules not processed for user `17c94d4500f94d4bb68f1379043830f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:28.813104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-973d6cd5-6014-42fa-8dae-78c7eb1323f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:28.876384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1193/2379] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:28.887663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e6850-5632-4ea7-b037-599bf8a0cff9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:28.889423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-757e6850-5632-4ea7-b037-599bf8a0cff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:28.945255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e6850-5632-4ea7-b037-599bf8a0cff9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:28.948943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a288c-b6df-45eb-90ae-e5aeccc87ba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:28.949578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a288c-b6df-45eb-90ae-e5aeccc87ba9 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:28.949880 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a288c-b6df-45eb-90ae-e5aeccc87ba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:28.950168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a288c-b6df-45eb-90ae-e5aeccc87ba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:28.960173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a288c-b6df-45eb-90ae-e5aeccc87ba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:28.961974 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a288c-b6df-45eb-90ae-e5aeccc87ba9 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:28.970856 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1187/2380] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:38.894667 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:38.966766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:39.360556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d6cd5-6014-42fa-8dae-78c7eb1323f7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:39.425179 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d6cd5-6014-42fa-8dae-78c7eb1323f7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:39.430734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:39.431364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:39.431754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:39.432057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] PATH_INFO: `/v3/users/17c94d4500f94d4bb68f1379043830f7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:39.433203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] RBAC: Authorizing `identity:delete_user(user_id=17c94d4500f94d4bb68f1379043830f7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:39.435121 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:39.652419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17c94d4500f94d4bb68f1379043830f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:39.652997 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17c94d4500f94d4bb68f1379043830f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:39.659772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17c94d4500f94d4bb68f1379043830f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:39.672128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17c94d4500f94d4bb68f1379043830f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:39.742032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17c94d4500f94d4bb68f1379043830f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:39.749060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1194/2381] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:47:39 2022] DELETE /identity/v3/users/17c94d4500f94d4bb68f1379043830f7 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:39.753338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fd90e-d6fd-41f9-9e77-699c26aaadb3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:39.760073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fd90e-d6fd-41f9-9e77-699c26aaadb3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:39.760073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fd90e-d6fd-41f9-9e77-699c26aaadb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:39.773731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f72fd90e-d6fd-41f9-9e77-699c26aaadb3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:39.774990 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f72fd90e-d6fd-41f9-9e77-699c26aaadb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:39.836556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1188/2382] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:39.846170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:39.847614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:39.903552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e233e5-9ae8-4ce5-9e3c-a21a387bf70c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:39.907252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ca6f5-b573-455a-9ef9-a03a42ad6cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:39.907885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ca6f5-b573-455a-9ef9-a03a42ad6cbe service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:39.908194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ca6f5-b573-455a-9ef9-a03a42ad6cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:39.908467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ca6f5-b573-455a-9ef9-a03a42ad6cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:39.917913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ca6f5-b573-455a-9ef9-a03a42ad6cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:39.919372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ca6f5-b573-455a-9ef9-a03a42ad6cbe service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:39.928787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1195/2383] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:40.458239 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fd90e-d6fd-41f9-9e77-699c26aaadb3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:40.460100 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f72fd90e-d6fd-41f9-9e77-699c26aaadb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:40.480583 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fd90e-d6fd-41f9-9e77-699c26aaadb3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:40.492449 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:40.493279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:40.493607 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:40.493885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] PATH_INFO: `/v3/projects/d458bafdab824cc1adad543dc2da149d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:40.495112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d458bafdab824cc1adad543dc2da149d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:40.497149 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:40.638814 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd458bafdab824cc1adad543dc2da149d', 'request_id': 'req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:40.645042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd458bafdab824cc1adad543dc2da149d', 'request_id': 'req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:40.764752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd458bafdab824cc1adad543dc2da149d', 'request_id': 'req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:40.776818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] The token cache is being invalidate because project d458bafdab824cc1adad543dc2da149d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:47:40.777212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:40.777960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7f12fc73-0e68-4a5c-b6e4-9aa9401a12c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:40.779479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1189/2384] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:47:40 2022] DELETE /identity/v3/projects/d458bafdab824cc1adad543dc2da149d => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:40.784373 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19e50-ebd9-4a0f-ba68-e86e2d62aaa3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:40.784657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19e50-ebd9-4a0f-ba68-e86e2d62aaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:40.784928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19e50-ebd9-4a0f-ba68-e86e2d62aaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:40.804297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-81a19e50-ebd9-4a0f-ba68-e86e2d62aaa3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:40.805466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-81a19e50-ebd9-4a0f-ba68-e86e2d62aaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:40.822101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:40.862922 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1196/2385] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:40.871504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37820518-cad4-4276-b316-125fab1f5b8e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:40.872182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37820518-cad4-4276-b316-125fab1f5b8e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:40.872182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37820518-cad4-4276-b316-125fab1f5b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:40.893526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-37820518-cad4-4276-b316-125fab1f5b8e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:40.894739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-37820518-cad4-4276-b316-125fab1f5b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:40.903444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1190/2386] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:40.907836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a19e50-ebd9-4a0f-ba68-e86e2d62aaa3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:40.918243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a19e50-ebd9-4a0f-ba68-e86e2d62aaa3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:40.922905 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3ba46-3dbc-442b-b574-428deb967786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:40.923452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3ba46-3dbc-442b-b574-428deb967786 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:40.923733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3ba46-3dbc-442b-b574-428deb967786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:40.924040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3ba46-3dbc-442b-b574-428deb967786 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:40.924734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3ba46-3dbc-442b-b574-428deb967786 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:47:40.925041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3ba46-3dbc-442b-b574-428deb967786 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:40.926685 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3ba46-3dbc-442b-b574-428deb967786 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:40.938613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1197/2387] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:47:40.943011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37820518-cad4-4276-b316-125fab1f5b8e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:40.952458 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37820518-cad4-4276-b316-125fab1f5b8e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:40.957500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853447eb-7ca3-4427-b45a-3392a0995c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:40.958115 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853447eb-7ca3-4427-b45a-3392a0995c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:40.958665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853447eb-7ca3-4427-b45a-3392a0995c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:40.958665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853447eb-7ca3-4427-b45a-3392a0995c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:40.959318 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853447eb-7ca3-4427-b45a-3392a0995c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:40.961249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853447eb-7ca3-4427-b45a-3392a0995c8b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:41.079148 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1191/2388] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:47:40 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:41.083623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3ba46-3dbc-442b-b574-428deb967786 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:41.093698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3ba46-3dbc-442b-b574-428deb967786 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:41.151405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947556d-13a7-4ab8-aaa4-8a33fae99aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:41.152042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947556d-13a7-4ab8-aaa4-8a33fae99aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:41.152323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947556d-13a7-4ab8-aaa4-8a33fae99aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:41.152601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947556d-13a7-4ab8-aaa4-8a33fae99aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:41.153257 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947556d-13a7-4ab8-aaa4-8a33fae99aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:41.155193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947556d-13a7-4ab8-aaa4-8a33fae99aa8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:41.246737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1198/2389] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:47:41 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:41.248263 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853447eb-7ca3-4427-b45a-3392a0995c8b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:41.258767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853447eb-7ca3-4427-b45a-3392a0995c8b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:41.263450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ef60f-bdcb-453b-a023-c776381a3126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:41.264133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ef60f-bdcb-453b-a023-c776381a3126 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:41.264409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ef60f-bdcb-453b-a023-c776381a3126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:41.269929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ef60f-bdcb-453b-a023-c776381a3126 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:41.269929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ef60f-bdcb-453b-a023-c776381a3126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:41.269929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ef60f-bdcb-453b-a023-c776381a3126 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:41.284912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1192/2390] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:41.286318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947556d-13a7-4ab8-aaa4-8a33fae99aa8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:41.297972 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947556d-13a7-4ab8-aaa4-8a33fae99aa8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:41.302578 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e71c2-7f75-4881-981b-bf145600e7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:41.303276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e71c2-7f75-4881-981b-bf145600e7cc admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:41.303558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e71c2-7f75-4881-981b-bf145600e7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:41.303832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e71c2-7f75-4881-981b-bf145600e7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:41.304555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e71c2-7f75-4881-981b-bf145600e7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:41.306153 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e71c2-7f75-4881-981b-bf145600e7cc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:41.316648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1199/2391] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:41.320922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ef60f-bdcb-453b-a023-c776381a3126 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:41.331146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ef60f-bdcb-453b-a023-c776381a3126 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:41.336486 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11f6ce-a2df-4ca7-843f-bddf6834f5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:41.337141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11f6ce-a2df-4ca7-843f-bddf6834f5a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:41.337425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11f6ce-a2df-4ca7-843f-bddf6834f5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:41.337702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11f6ce-a2df-4ca7-843f-bddf6834f5a2 admin admin] PATH_INFO: `/v3/projects/3f67822710044f518ee55d1fef35ced2/users/2d6a8551f7bd4ced9c10646601e5fde1/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:41.352694 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11f6ce-a2df-4ca7-843f-bddf6834f5a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f67822710044f518ee55d1fef35ced2, user_id=2d6a8551f7bd4ced9c10646601e5fde1, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:41.355332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11f6ce-a2df-4ca7-843f-bddf6834f5a2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:41.443630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1193/2392] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:41 2022] PUT /identity/v3/projects/3f67822710044f518ee55d1fef35ced2/users/2d6a8551f7bd4ced9c10646601e5fde1/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:47:41.454253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6fe2-7cbe-443a-b16d-542e708a4e38 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:41.454560 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6fe2-7cbe-443a-b16d-542e708a4e38 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:41.454834 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6fe2-7cbe-443a-b16d-542e708a4e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:41.485961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c84d6fe2-7cbe-443a-b16d-542e708a4e38 None None] MFA Rules not processed for user `2d6a8551f7bd4ced9c10646601e5fde1`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:41.487277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c84d6fe2-7cbe-443a-b16d-542e708a4e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:41.554802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1200/2393] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:41.564976 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11f6ce-a2df-4ca7-843f-bddf6834f5a2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:41.587515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:41.628821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11f6ce-a2df-4ca7-843f-bddf6834f5a2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:41.633621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b172fe1-6374-4017-b73e-66f974e483a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:41.634236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b172fe1-6374-4017-b73e-66f974e483a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:41.634524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b172fe1-6374-4017-b73e-66f974e483a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:41.634828 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b172fe1-6374-4017-b73e-66f974e483a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:41.635504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b172fe1-6374-4017-b73e-66f974e483a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:41.638403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b172fe1-6374-4017-b73e-66f974e483a5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:41.737632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1194/2394] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:47:41 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:41.742690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d6fe2-7cbe-443a-b16d-542e708a4e38 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:41.753740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d6fe2-7cbe-443a-b16d-542e708a4e38 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:41.765991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:41.810040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669aa5d-0913-4750-ae36-e9b1167a0d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:41.810647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669aa5d-0913-4750-ae36-e9b1167a0d10 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:41.810932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669aa5d-0913-4750-ae36-e9b1167a0d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:41.811206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669aa5d-0913-4750-ae36-e9b1167a0d10 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:41.811864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669aa5d-0913-4750-ae36-e9b1167a0d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:41.813956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669aa5d-0913-4750-ae36-e9b1167a0d10 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:41.908028 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1201/2395] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:47:41 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:41.912595 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b172fe1-6374-4017-b73e-66f974e483a5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:41.923211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b172fe1-6374-4017-b73e-66f974e483a5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:41.928916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ee12e-65fc-4305-a42f-91a245d4e588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:41.929658 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ee12e-65fc-4305-a42f-91a245d4e588 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:41.929949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ee12e-65fc-4305-a42f-91a245d4e588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:41.930227 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ee12e-65fc-4305-a42f-91a245d4e588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:41.930917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ee12e-65fc-4305-a42f-91a245d4e588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:41.932590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ee12e-65fc-4305-a42f-91a245d4e588 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:41.943262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1195/2396] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:41.947574 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669aa5d-0913-4750-ae36-e9b1167a0d10 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:41.958145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669aa5d-0913-4750-ae36-e9b1167a0d10 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:41.962817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a8bd8-7325-4a69-8faa-8c7441ffd705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:41.963483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a8bd8-7325-4a69-8faa-8c7441ffd705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:41.963766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a8bd8-7325-4a69-8faa-8c7441ffd705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:41.964082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a8bd8-7325-4a69-8faa-8c7441ffd705 admin admin] PATH_INFO: `/v3/projects/a6f4a7a9a4d24a3992fd4cd5506d5fd7/users/217c57cf74164fb19158be32f12cc104/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:41.979358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a8bd8-7325-4a69-8faa-8c7441ffd705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f4a7a9a4d24a3992fd4cd5506d5fd7, user_id=217c57cf74164fb19158be32f12cc104, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:41.982051 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a8bd8-7325-4a69-8faa-8c7441ffd705 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:42.096611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1202/2397] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:41 2022] PUT /identity/v3/projects/a6f4a7a9a4d24a3992fd4cd5506d5fd7/users/217c57cf74164fb19158be32f12cc104/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:42.100854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ee12e-65fc-4305-a42f-91a245d4e588 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:42.112272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ee12e-65fc-4305-a42f-91a245d4e588 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:42.171074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be85d5f-894d-4ca2-bb59-dae8feebc874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:42.171690 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be85d5f-894d-4ca2-bb59-dae8feebc874 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:42.172053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be85d5f-894d-4ca2-bb59-dae8feebc874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:42.172334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be85d5f-894d-4ca2-bb59-dae8feebc874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:42.173022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be85d5f-894d-4ca2-bb59-dae8feebc874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:42.174601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be85d5f-894d-4ca2-bb59-dae8feebc874 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:42.184939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1196/2398] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:42.189176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a8bd8-7325-4a69-8faa-8c7441ffd705 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:42.200252 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a8bd8-7325-4a69-8faa-8c7441ffd705 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:42.205187 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242bf65-b3f3-4598-85fe-214ac5653db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:42.205843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242bf65-b3f3-4598-85fe-214ac5653db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:42.206126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242bf65-b3f3-4598-85fe-214ac5653db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:42.206400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242bf65-b3f3-4598-85fe-214ac5653db7 admin admin] PATH_INFO: `/v3/projects/a6f4a7a9a4d24a3992fd4cd5506d5fd7/users/217c57cf74164fb19158be32f12cc104/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:42.208420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242bf65-b3f3-4598-85fe-214ac5653db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f4a7a9a4d24a3992fd4cd5506d5fd7, user_id=217c57cf74164fb19158be32f12cc104, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:42.211065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242bf65-b3f3-4598-85fe-214ac5653db7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:42.221195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1203/2399] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:42 2022] PUT /identity/v3/projects/a6f4a7a9a4d24a3992fd4cd5506d5fd7/users/217c57cf74164fb19158be32f12cc104/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:42.232371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965659-542a-4b38-a7fb-738925f8aa99 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:42.232662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965659-542a-4b38-a7fb-738925f8aa99 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:42.232940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965659-542a-4b38-a7fb-738925f8aa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:42.261202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d8965659-542a-4b38-a7fb-738925f8aa99 None None] MFA Rules not processed for user `217c57cf74164fb19158be32f12cc104`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:42.262403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d8965659-542a-4b38-a7fb-738925f8aa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:42.334183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1197/2400] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:42.345357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242bf65-b3f3-4598-85fe-214ac5653db7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:42.347456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3242bf65-b3f3-4598-85fe-214ac5653db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:42.407828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242bf65-b3f3-4598-85fe-214ac5653db7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:42.411439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2672a5-86e0-45df-a582-4e45f42b45b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:42.412700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2672a5-86e0-45df-a582-4e45f42b45b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:42.412981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2672a5-86e0-45df-a582-4e45f42b45b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:42.413255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2672a5-86e0-45df-a582-4e45f42b45b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:42.475454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2672a5-86e0-45df-a582-4e45f42b45b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:42.476935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2672a5-86e0-45df-a582-4e45f42b45b5 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:42.486361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1204/2401] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:43.835905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8965659-542a-4b38-a7fb-738925f8aa99 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:43.837468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d8965659-542a-4b38-a7fb-738925f8aa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:43.859049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:44.012013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8965659-542a-4b38-a7fb-738925f8aa99 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:44.018498 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3bca8-af29-4a83-b24e-82ccee1808c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:44.019236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3bca8-af29-4a83-b24e-82ccee1808c5 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:44.019530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3bca8-af29-4a83-b24e-82ccee1808c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:44.019810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3bca8-af29-4a83-b24e-82ccee1808c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:44.022919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8bb3bca8-af29-4a83-b24e-82ccee1808c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:44.037407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3bca8-af29-4a83-b24e-82ccee1808c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:44.039851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3bca8-af29-4a83-b24e-82ccee1808c5 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:44.051865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1198/2402] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:45.806368 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2672a5-86e0-45df-a582-4e45f42b45b5 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:45.833761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:45.870073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2672a5-86e0-45df-a582-4e45f42b45b5 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:45.874692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:45.875342 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:45.875625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:45.875902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] PATH_INFO: `/v3/users/2d6a8551f7bd4ced9c10646601e5fde1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:45.877105 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d6a8551f7bd4ced9c10646601e5fde1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:45.879047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:46.110259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d6a8551f7bd4ced9c10646601e5fde1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:46.110906 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d6a8551f7bd4ced9c10646601e5fde1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:46.120533 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d6a8551f7bd4ced9c10646601e5fde1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:46.136212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d6a8551f7bd4ced9c10646601e5fde1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:46.212365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d6a8551f7bd4ced9c10646601e5fde1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:46.220388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1205/2403] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:47:45 2022] DELETE /identity/v3/users/2d6a8551f7bd4ced9c10646601e5fde1 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:46.224277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40c712-e1e1-47dc-ae8e-8eb3b5ca16da None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:46.224768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40c712-e1e1-47dc-ae8e-8eb3b5ca16da None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:46.225231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40c712-e1e1-47dc-ae8e-8eb3b5ca16da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:46.248740 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9e40c712-e1e1-47dc-ae8e-8eb3b5ca16da None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:46.250072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9e40c712-e1e1-47dc-ae8e-8eb3b5ca16da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:46.319765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1199/2404] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:47:46.341480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:46.343078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:46.422274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f58ca-a652-42a7-9ef9-71c76e320048 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:46.425989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0442331-2703-4305-9f2e-50420a4e0d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:46.426610 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0442331-2703-4305-9f2e-50420a4e0d5b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:46.426923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0442331-2703-4305-9f2e-50420a4e0d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:46.427154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0442331-2703-4305-9f2e-50420a4e0d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:46.439740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0442331-2703-4305-9f2e-50420a4e0d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:46.441315 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0442331-2703-4305-9f2e-50420a4e0d5b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:46.456004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1206/2405] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:46.964931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40c712-e1e1-47dc-ae8e-8eb3b5ca16da None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:46.964931 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9e40c712-e1e1-47dc-ae8e-8eb3b5ca16da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:46.969779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40c712-e1e1-47dc-ae8e-8eb3b5ca16da None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:46.974585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:46.975304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:46.976283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:46.976639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] PATH_INFO: `/v3/projects/3f67822710044f518ee55d1fef35ced2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:46.977925 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f67822710044f518ee55d1fef35ced2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:46.980329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:47.112407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f67822710044f518ee55d1fef35ced2', 'request_id': 'req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:47.119454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f67822710044f518ee55d1fef35ced2', 'request_id': 'req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:47.226411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f67822710044f518ee55d1fef35ced2', 'request_id': 'req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:47.237196 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] The token cache is being invalidate because project 3f67822710044f518ee55d1fef35ced2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:47:47.237488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:47.238112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:47.239622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1200/2406] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:47:46 2022] DELETE /identity/v3/projects/3f67822710044f518ee55d1fef35ced2 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:47.244517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0442331-2703-4305-9f2e-50420a4e0d5b service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:47.254030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0442331-2703-4305-9f2e-50420a4e0d5b service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:47.297479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:47.310553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ab5cf-ab8a-4377-92f7-d3244dfa25f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:47.311130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ab5cf-ab8a-4377-92f7-d3244dfa25f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:47.311413 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ab5cf-ab8a-4377-92f7-d3244dfa25f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:47.311691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ab5cf-ab8a-4377-92f7-d3244dfa25f9 admin admin] PATH_INFO: `/v3/users/217c57cf74164fb19158be32f12cc104` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:47.312879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ab5cf-ab8a-4377-92f7-d3244dfa25f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=217c57cf74164fb19158be32f12cc104)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:47.314839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ab5cf-ab8a-4377-92f7-d3244dfa25f9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:47.597330 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-367ab5cf-ab8a-4377-92f7-d3244dfa25f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '217c57cf74164fb19158be32f12cc104'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:47.598354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-367ab5cf-ab8a-4377-92f7-d3244dfa25f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '217c57cf74164fb19158be32f12cc104'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:47.612263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-367ab5cf-ab8a-4377-92f7-d3244dfa25f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '217c57cf74164fb19158be32f12cc104'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:47.624947 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-367ab5cf-ab8a-4377-92f7-d3244dfa25f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '217c57cf74164fb19158be32f12cc104'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:47.757247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-367ab5cf-ab8a-4377-92f7-d3244dfa25f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '217c57cf74164fb19158be32f12cc104'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:47.763559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1207/2407] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:47:47 2022] DELETE /identity/v3/users/217c57cf74164fb19158be32f12cc104 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:48.224135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:48.225585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:48.244384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83b41c-20d1-40a6-b53e-6f56b39c24b8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:48.265960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:48.301810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:48.302459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:48.302761 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:48.303044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] PATH_INFO: `/v3/projects/a6f4a7a9a4d24a3992fd4cd5506d5fd7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:48.304361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f4a7a9a4d24a3992fd4cd5506d5fd7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:48.306288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:48.468336 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f4a7a9a4d24a3992fd4cd5506d5fd7', 'request_id': 'req-7c553b73-9014-4a0a-9d48-8f94bef65b35'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:48.474439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f4a7a9a4d24a3992fd4cd5506d5fd7', 'request_id': 'req-7c553b73-9014-4a0a-9d48-8f94bef65b35'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:48.612688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f4a7a9a4d24a3992fd4cd5506d5fd7', 'request_id': 'req-7c553b73-9014-4a0a-9d48-8f94bef65b35'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:48.636062 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] The token cache is being invalidate because project a6f4a7a9a4d24a3992fd4cd5506d5fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:47:48.636774 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:48.637460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7c553b73-9014-4a0a-9d48-8f94bef65b35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:48.639592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1201/2408] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:47:48 2022] DELETE /identity/v3/projects/a6f4a7a9a4d24a3992fd4cd5506d5fd7 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:48.644835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127191e-fe5b-4315-bad0-c038ec2cfd0a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:48.648513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127191e-fe5b-4315-bad0-c038ec2cfd0a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:48.648513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127191e-fe5b-4315-bad0-c038ec2cfd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:48.669384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1127191e-fe5b-4315-bad0-c038ec2cfd0a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:48.670611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1127191e-fe5b-4315-bad0-c038ec2cfd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:48.734770 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1208/2409] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:48.743465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df1b9-953c-46b1-8b4b-c7d3189fa819 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:48.745875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df1b9-953c-46b1-8b4b-c7d3189fa819 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:48.745875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df1b9-953c-46b1-8b4b-c7d3189fa819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:48.763614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-086df1b9-953c-46b1-8b4b-c7d3189fa819 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:48.764874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-086df1b9-953c-46b1-8b4b-c7d3189fa819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:48.773192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1202/2410] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:47:48.777371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127191e-fe5b-4315-bad0-c038ec2cfd0a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:48.787212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127191e-fe5b-4315-bad0-c038ec2cfd0a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:48.792797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e82a05-35f1-4f90-9db5-60756a56abd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:48.793416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e82a05-35f1-4f90-9db5-60756a56abd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:48.793697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e82a05-35f1-4f90-9db5-60756a56abd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:48.793972 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e82a05-35f1-4f90-9db5-60756a56abd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:48.794663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e82a05-35f1-4f90-9db5-60756a56abd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:47:48.794991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e82a05-35f1-4f90-9db5-60756a56abd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:48.796627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e82a05-35f1-4f90-9db5-60756a56abd9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:48.808879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1209/2411] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:47:48.813171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086df1b9-953c-46b1-8b4b-c7d3189fa819 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:48.822768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086df1b9-953c-46b1-8b4b-c7d3189fa819 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:48.827566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb88b0-484b-45d0-9ec9-4956792f2867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:48.828216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb88b0-484b-45d0-9ec9-4956792f2867 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:48.828525 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb88b0-484b-45d0-9ec9-4956792f2867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:48.829059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb88b0-484b-45d0-9ec9-4956792f2867 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:48.829735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb88b0-484b-45d0-9ec9-4956792f2867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:48.831643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb88b0-484b-45d0-9ec9-4956792f2867 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:49.029616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1203/2412] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:47:48 2022] POST /identity/v3/projects => generated 393 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:49.033980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e82a05-35f1-4f90-9db5-60756a56abd9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:49.050259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e82a05-35f1-4f90-9db5-60756a56abd9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:49.094874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:49.104399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ea21c-220d-448b-abba-85db2635af1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:49.105045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ea21c-220d-448b-abba-85db2635af1b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:49.105328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ea21c-220d-448b-abba-85db2635af1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:49.109854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ea21c-220d-448b-abba-85db2635af1b admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:49.109854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ea21c-220d-448b-abba-85db2635af1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:49.109854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ea21c-220d-448b-abba-85db2635af1b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:49.227722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1210/2413] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:47:49 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:49.231937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb88b0-484b-45d0-9ec9-4956792f2867 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:49.242989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb88b0-484b-45d0-9ec9-4956792f2867 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:49.247656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464272e-2de5-4457-a15f-f2625e4503d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:49.248312 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464272e-2de5-4457-a15f-f2625e4503d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:49.248590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464272e-2de5-4457-a15f-f2625e4503d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:49.248861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464272e-2de5-4457-a15f-f2625e4503d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:49.249555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464272e-2de5-4457-a15f-f2625e4503d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:49.251127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464272e-2de5-4457-a15f-f2625e4503d6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:49.261731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1204/2414] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:49.265763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ea21c-220d-448b-abba-85db2635af1b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:49.276391 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ea21c-220d-448b-abba-85db2635af1b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:49.281019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7186e-4347-45f2-b93e-6847c37f0e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:49.281670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7186e-4347-45f2-b93e-6847c37f0e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:49.281913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7186e-4347-45f2-b93e-6847c37f0e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:49.282253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7186e-4347-45f2-b93e-6847c37f0e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:49.282931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7186e-4347-45f2-b93e-6847c37f0e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:49.284566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7186e-4347-45f2-b93e-6847c37f0e39 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:49.296897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1211/2415] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:47:49.301184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464272e-2de5-4457-a15f-f2625e4503d6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:49.311800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464272e-2de5-4457-a15f-f2625e4503d6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:49.319241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d832e-6c34-436e-87eb-e1818c1c92e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:49.319241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d832e-6c34-436e-87eb-e1818c1c92e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:49.319241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d832e-6c34-436e-87eb-e1818c1c92e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:49.319241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d832e-6c34-436e-87eb-e1818c1c92e5 admin admin] PATH_INFO: `/v3/projects/8548962986344cf5a4b34ab004e5ce88/users/2c2b47da3e174daebf5332ebf61026fd/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:49.336223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d832e-6c34-436e-87eb-e1818c1c92e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8548962986344cf5a4b34ab004e5ce88, user_id=2c2b47da3e174daebf5332ebf61026fd, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:49.339262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d832e-6c34-436e-87eb-e1818c1c92e5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:49.453338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1205/2416] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:49 2022] PUT /identity/v3/projects/8548962986344cf5a4b34ab004e5ce88/users/2c2b47da3e174daebf5332ebf61026fd/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:49.464772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba3aa9-cc9b-4309-9a6c-ad25c3e7afd9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:49.465069 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba3aa9-cc9b-4309-9a6c-ad25c3e7afd9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:49.465372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba3aa9-cc9b-4309-9a6c-ad25c3e7afd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:49.522859 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-48ba3aa9-cc9b-4309-9a6c-ad25c3e7afd9 None None] MFA Rules not processed for user `2c2b47da3e174daebf5332ebf61026fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:49.524203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-48ba3aa9-cc9b-4309-9a6c-ad25c3e7afd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:49.600383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1212/2417] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:49.607700 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d832e-6c34-436e-87eb-e1818c1c92e5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:49.609498 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c21d832e-6c34-436e-87eb-e1818c1c92e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:49.674360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d832e-6c34-436e-87eb-e1818c1c92e5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:49.678048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca9982-4f7a-45a7-8b48-142bf6b90f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:49.678711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca9982-4f7a-45a7-8b48-142bf6b90f7f service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:49.679044 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca9982-4f7a-45a7-8b48-142bf6b90f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:49.679489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca9982-4f7a-45a7-8b48-142bf6b90f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:49.691293 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:49.697628 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca9982-4f7a-45a7-8b48-142bf6b90f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:49.699102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca9982-4f7a-45a7-8b48-142bf6b90f7f service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:49.710766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1206/2418] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:49 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:49.937507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f57ee-bc12-46e7-8f2c-5505f94fe715 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:49.937984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f57ee-bc12-46e7-8f2c-5505f94fe715 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:49.938211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f57ee-bc12-46e7-8f2c-5505f94fe715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:49.960546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-348f57ee-bc12-46e7-8f2c-5505f94fe715 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:49.961979 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-348f57ee-bc12-46e7-8f2c-5505f94fe715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:50.019168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1213/2419] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:47:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:50.163428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca9982-4f7a-45a7-8b48-142bf6b90f7f service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:50.164719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b3ca9982-4f7a-45a7-8b48-142bf6b90f7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:50.239010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca9982-4f7a-45a7-8b48-142bf6b90f7f service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:50.244310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469261b6-b9ab-42a5-bc20-b8d732ab717e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:50.244929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469261b6-b9ab-42a5-bc20-b8d732ab717e service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:50.246661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469261b6-b9ab-42a5-bc20-b8d732ab717e service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:50.246661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469261b6-b9ab-42a5-bc20-b8d732ab717e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:50.253516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469261b6-b9ab-42a5-bc20-b8d732ab717e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:50.255117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469261b6-b9ab-42a5-bc20-b8d732ab717e service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:50.263767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1207/2420] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:47:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:47:50.831304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f57ee-bc12-46e7-8f2c-5505f94fe715 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:50.832921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-348f57ee-bc12-46e7-8f2c-5505f94fe715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:50.910464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f57ee-bc12-46e7-8f2c-5505f94fe715 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:50.914705 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e9a05-8738-4d79-9de8-43adfaca6227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:50.915603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9a05-8738-4d79-9de8-43adfaca6227 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:50.916110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9a05-8738-4d79-9de8-43adfaca6227 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:50.916393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9a05-8738-4d79-9de8-43adfaca6227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:50.918231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-022e9a05-8738-4d79-9de8-43adfaca6227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:50.963611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:51.017489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e9a05-8738-4d79-9de8-43adfaca6227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:51.019229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e9a05-8738-4d79-9de8-43adfaca6227 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:51.034136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1214/2421] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:51.282262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469261b6-b9ab-42a5-bc20-b8d732ab717e service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:51.285518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-469261b6-b9ab-42a5-bc20-b8d732ab717e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:51.436911 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469261b6-b9ab-42a5-bc20-b8d732ab717e service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:51.443908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab41c67-dc4a-4b88-8569-437d2dd2711c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:51.445664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab41c67-dc4a-4b88-8569-437d2dd2711c service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:51.446189 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab41c67-dc4a-4b88-8569-437d2dd2711c service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:51.446678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab41c67-dc4a-4b88-8569-437d2dd2711c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:51.456817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7ab41c67-dc4a-4b88-8569-437d2dd2711c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:51.484099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab41c67-dc4a-4b88-8569-437d2dd2711c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:51.485757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab41c67-dc4a-4b88-8569-437d2dd2711c service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:51.515285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1208/2422] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:55.894448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e9a05-8738-4d79-9de8-43adfaca6227 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:55.896313 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-022e9a05-8738-4d79-9de8-43adfaca6227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:55.997205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e9a05-8738-4d79-9de8-43adfaca6227 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:56.004152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798daf1-a9dd-4304-8f6f-0978a057702a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:56.005105 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798daf1-a9dd-4304-8f6f-0978a057702a service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:56.005586 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798daf1-a9dd-4304-8f6f-0978a057702a service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:56.006082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798daf1-a9dd-4304-8f6f-0978a057702a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:56.008425 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4798daf1-a9dd-4304-8f6f-0978a057702a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:56.024273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798daf1-a9dd-4304-8f6f-0978a057702a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:56.026914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798daf1-a9dd-4304-8f6f-0978a057702a service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:56.040802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1215/2423] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:56.087913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab41c67-dc4a-4b88-8569-437d2dd2711c service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:56.089687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7ab41c67-dc4a-4b88-8569-437d2dd2711c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:56.106477 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab41c67-dc4a-4b88-8569-437d2dd2711c service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:56.114381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a4ab7-3de4-41be-aa61-db9ab9bcdb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:56.115120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4ab7-3de4-41be-aa61-db9ab9bcdb7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:56.116462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4ab7-3de4-41be-aa61-db9ab9bcdb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:56.116462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4ab7-3de4-41be-aa61-db9ab9bcdb7a admin admin] PATH_INFO: `/v3/users/e44a63323cfb40a2bb7a5e83d8636f38` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:56.118156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a4ab7-3de4-41be-aa61-db9ab9bcdb7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e44a63323cfb40a2bb7a5e83d8636f38)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:56.120512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a4ab7-3de4-41be-aa61-db9ab9bcdb7a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:56.363534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2e1a4ab7-3de4-41be-aa61-db9ab9bcdb7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e44a63323cfb40a2bb7a5e83d8636f38'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:56.364569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2e1a4ab7-3de4-41be-aa61-db9ab9bcdb7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e44a63323cfb40a2bb7a5e83d8636f38'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:56.365406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2e1a4ab7-3de4-41be-aa61-db9ab9bcdb7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e44a63323cfb40a2bb7a5e83d8636f38'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:56.385875 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:56.388951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2e1a4ab7-3de4-41be-aa61-db9ab9bcdb7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e44a63323cfb40a2bb7a5e83d8636f38'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:56.494792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2e1a4ab7-3de4-41be-aa61-db9ab9bcdb7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e44a63323cfb40a2bb7a5e83d8636f38'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:56.519214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1209/2424] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:47:56 2022] DELETE /identity/v3/users/e44a63323cfb40a2bb7a5e83d8636f38 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:56.860013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798daf1-a9dd-4304-8f6f-0978a057702a service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:56.888043 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798daf1-a9dd-4304-8f6f-0978a057702a service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:56.957057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:56.957769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:56.958158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:56.958657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] PATH_INFO: `/v3/projects/3397da2ce7ff494389f4134624f3a659` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:56.960771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3397da2ce7ff494389f4134624f3a659)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:56.962767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:57.093751 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:57.095864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3397da2ce7ff494389f4134624f3a659', 'request_id': 'req-08e767f7-5249-4648-8fc8-aed198a5f901'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:57.541654 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3397da2ce7ff494389f4134624f3a659', 'request_id': 'req-08e767f7-5249-4648-8fc8-aed198a5f901'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:57.562422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3397da2ce7ff494389f4134624f3a659', 'request_id': 'req-08e767f7-5249-4648-8fc8-aed198a5f901'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:57.571452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] The token cache is being invalidate because project 3397da2ce7ff494389f4134624f3a659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:47:57.571742 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:57.572436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e767f7-5249-4648-8fc8-aed198a5f901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:57.574229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1216/2425] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:47:56 2022] DELETE /identity/v3/projects/3397da2ce7ff494389f4134624f3a659 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:57.580982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d646e-4abf-4344-b14f-a5fb6478a076 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:57.581841 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d646e-4abf-4344-b14f-a5fb6478a076 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:57.582129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d646e-4abf-4344-b14f-a5fb6478a076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:57.604344 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5d3d646e-4abf-4344-b14f-a5fb6478a076 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:57.605723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5d3d646e-4abf-4344-b14f-a5fb6478a076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:57.688129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1210/2426] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:57.698696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04085b4-95cb-4f91-9527-942c75df713c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:57.698983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04085b4-95cb-4f91-9527-942c75df713c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:57.699253 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04085b4-95cb-4f91-9527-942c75df713c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:57.721584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d04085b4-95cb-4f91-9527-942c75df713c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:57.722939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d04085b4-95cb-4f91-9527-942c75df713c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:57.743679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1217/2427] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:47:57.744950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d646e-4abf-4344-b14f-a5fb6478a076 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:57.760075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d646e-4abf-4344-b14f-a5fb6478a076 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:57.764907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5005a82-59c6-41eb-9c4d-8839aa88152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:57.765618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5005a82-59c6-41eb-9c4d-8839aa88152a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:57.765970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5005a82-59c6-41eb-9c4d-8839aa88152a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:57.766312 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5005a82-59c6-41eb-9c4d-8839aa88152a admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:57.767271 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5005a82-59c6-41eb-9c4d-8839aa88152a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:47:57.767795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5005a82-59c6-41eb-9c4d-8839aa88152a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:57.769489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5005a82-59c6-41eb-9c4d-8839aa88152a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:57.790981 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1211/2428] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:47:57.792259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04085b4-95cb-4f91-9527-942c75df713c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:57.803595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04085b4-95cb-4f91-9527-942c75df713c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:57.809966 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9a62-bc60-45ed-9b8b-a068b9b78725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:57.815452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9a62-bc60-45ed-9b8b-a068b9b78725 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:57.815452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9a62-bc60-45ed-9b8b-a068b9b78725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:57.815452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9a62-bc60-45ed-9b8b-a068b9b78725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:57.815452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e9a62-bc60-45ed-9b8b-a068b9b78725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:57.815452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e9a62-bc60-45ed-9b8b-a068b9b78725 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:57.910639 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1218/2429] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:47:57 2022] POST /identity/v3/projects => generated 393 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:57.915123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5005a82-59c6-41eb-9c4d-8839aa88152a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:57.926388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5005a82-59c6-41eb-9c4d-8839aa88152a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:58.015053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b37537-ee3d-4916-b26a-0beb0408ad3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:58.015685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b37537-ee3d-4916-b26a-0beb0408ad3a admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:58.016042 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b37537-ee3d-4916-b26a-0beb0408ad3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:58.016317 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b37537-ee3d-4916-b26a-0beb0408ad3a admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:58.017152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b37537-ee3d-4916-b26a-0beb0408ad3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:58.020253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b37537-ee3d-4916-b26a-0beb0408ad3a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:58.140753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1212/2430] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:47:57 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:58.144294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9a62-bc60-45ed-9b8b-a068b9b78725 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:58.161010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9a62-bc60-45ed-9b8b-a068b9b78725 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:58.172206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8965a2d-6600-445c-a95c-3e1133477f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:58.184311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8965a2d-6600-445c-a95c-3e1133477f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:58.186168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8965a2d-6600-445c-a95c-3e1133477f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:58.186168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8965a2d-6600-445c-a95c-3e1133477f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:58.186168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8965a2d-6600-445c-a95c-3e1133477f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:58.187916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8965a2d-6600-445c-a95c-3e1133477f44 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:58.202686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1219/2431] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:58.204443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b37537-ee3d-4916-b26a-0beb0408ad3a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:58.217629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b37537-ee3d-4916-b26a-0beb0408ad3a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:58.222698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34cb02-4f44-4025-be3a-096f816e47fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:58.223378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34cb02-4f44-4025-be3a-096f816e47fa admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:58.223667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34cb02-4f44-4025-be3a-096f816e47fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:58.226706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34cb02-4f44-4025-be3a-096f816e47fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:58.226706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34cb02-4f44-4025-be3a-096f816e47fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:58.227344 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34cb02-4f44-4025-be3a-096f816e47fa admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:58.244032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1213/2432] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:58.246061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8965a2d-6600-445c-a95c-3e1133477f44 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:58.258353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8965a2d-6600-445c-a95c-3e1133477f44 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:58.263261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150869d-1152-467d-846d-80622cdef842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:58.263930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150869d-1152-467d-846d-80622cdef842 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:58.264283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150869d-1152-467d-846d-80622cdef842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:58.264593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150869d-1152-467d-846d-80622cdef842 admin admin] PATH_INFO: `/v3/projects/27f2406f6d8d45a0adba69c10261478e/users/b24da326dc0e4e52ae9c0c5825575f60/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:58.282894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150869d-1152-467d-846d-80622cdef842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f2406f6d8d45a0adba69c10261478e, user_id=b24da326dc0e4e52ae9c0c5825575f60, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:58.285650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150869d-1152-467d-846d-80622cdef842 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:58.411008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1220/2433] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:58 2022] PUT /identity/v3/projects/27f2406f6d8d45a0adba69c10261478e/users/b24da326dc0e4e52ae9c0c5825575f60/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:58.423619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd758d82-b2db-43b3-b71a-bb0d80ee6dbe None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:58.423917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd758d82-b2db-43b3-b71a-bb0d80ee6dbe None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:58.424237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd758d82-b2db-43b3-b71a-bb0d80ee6dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:58.457039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dd758d82-b2db-43b3-b71a-bb0d80ee6dbe None None] MFA Rules not processed for user `b24da326dc0e4e52ae9c0c5825575f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:58.458348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dd758d82-b2db-43b3-b71a-bb0d80ee6dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:58.498440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:58.539800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1214/2434] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:58.547613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150869d-1152-467d-846d-80622cdef842 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:58.624132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150869d-1152-467d-846d-80622cdef842 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:58.629562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e05cb-8cda-4f57-9050-3936cc6f097c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:58.630379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e05cb-8cda-4f57-9050-3936cc6f097c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:58.630841 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e05cb-8cda-4f57-9050-3936cc6f097c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:58.631326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e05cb-8cda-4f57-9050-3936cc6f097c admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:58.633412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e05cb-8cda-4f57-9050-3936cc6f097c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:58.635566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e05cb-8cda-4f57-9050-3936cc6f097c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:58.825764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd758d82-b2db-43b3-b71a-bb0d80ee6dbe None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:58.832350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1221/2435] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:47:58 2022] POST /identity/v3/projects => generated 391 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:47:58.842418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd758d82-b2db-43b3-b71a-bb0d80ee6dbe None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:58.917841 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f85453-69de-4ab6-93b2-471be700369c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:58.918391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f85453-69de-4ab6-93b2-471be700369c admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:58.918698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f85453-69de-4ab6-93b2-471be700369c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:58.918982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f85453-69de-4ab6-93b2-471be700369c admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:58.919649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f85453-69de-4ab6-93b2-471be700369c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:58.921633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f85453-69de-4ab6-93b2-471be700369c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:59.072809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1215/2436] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:47:58 2022] POST /identity/v3/users => generated 302 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:47:59.079378 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e05cb-8cda-4f57-9050-3936cc6f097c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:59.089916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e05cb-8cda-4f57-9050-3936cc6f097c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:59.094925 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d3a43-0c06-442d-a3c8-119f71d4478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:59.095815 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d3a43-0c06-442d-a3c8-119f71d4478a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:59.096261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d3a43-0c06-442d-a3c8-119f71d4478a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:59.096553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d3a43-0c06-442d-a3c8-119f71d4478a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:59.097245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d3a43-0c06-442d-a3c8-119f71d4478a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:59.098858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d3a43-0c06-442d-a3c8-119f71d4478a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:59.118788 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1222/2437] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:59.120078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f85453-69de-4ab6-93b2-471be700369c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:59.137426 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f85453-69de-4ab6-93b2-471be700369c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:59.144338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958e827-2c89-4d65-b895-d151fd189992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:59.145163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958e827-2c89-4d65-b895-d151fd189992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:59.145644 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958e827-2c89-4d65-b895-d151fd189992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:59.146040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958e827-2c89-4d65-b895-d151fd189992 admin admin] PATH_INFO: `/v3/projects/35c5594ff0d34b35af8139dc98ed1c5d/users/9a6159de6d62496ca664579e6a1d7d7a/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:59.165387 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958e827-2c89-4d65-b895-d151fd189992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c5594ff0d34b35af8139dc98ed1c5d, user_id=9a6159de6d62496ca664579e6a1d7d7a, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:59.168377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958e827-2c89-4d65-b895-d151fd189992 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:59.227891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d3a43-0c06-442d-a3c8-119f71d4478a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:59.233061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d81d3a43-0c06-442d-a3c8-119f71d4478a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:59.249729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d3a43-0c06-442d-a3c8-119f71d4478a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:59.259291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f842a3f-f246-49d8-b16b-094dc54ef01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:59.260954 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f842a3f-f246-49d8-b16b-094dc54ef01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:59.261522 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f842a3f-f246-49d8-b16b-094dc54ef01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:59.266241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f842a3f-f246-49d8-b16b-094dc54ef01b admin admin] PATH_INFO: `/v3/users/2c2b47da3e174daebf5332ebf61026fd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:59.266241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f842a3f-f246-49d8-b16b-094dc54ef01b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c2b47da3e174daebf5332ebf61026fd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:59.266518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f842a3f-f246-49d8-b16b-094dc54ef01b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:59.267687 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1216/2438] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:59 2022] PUT /identity/v3/projects/35c5594ff0d34b35af8139dc98ed1c5d/users/9a6159de6d62496ca664579e6a1d7d7a/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:59.271594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958e827-2c89-4d65-b895-d151fd189992 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:59.357815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958e827-2c89-4d65-b895-d151fd189992 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:59.362852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4882f9-ce50-4d2c-b4af-2eb44c8dee6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:59.363476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4882f9-ce50-4d2c-b4af-2eb44c8dee6e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:59.363762 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4882f9-ce50-4d2c-b4af-2eb44c8dee6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:59.364068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4882f9-ce50-4d2c-b4af-2eb44c8dee6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:59.364771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4882f9-ce50-4d2c-b4af-2eb44c8dee6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:59.366439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4882f9-ce50-4d2c-b4af-2eb44c8dee6e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:59.383031 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1217/2439] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:47:59.384189 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4882f9-ce50-4d2c-b4af-2eb44c8dee6e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:59.396118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4882f9-ce50-4d2c-b4af-2eb44c8dee6e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:59.401465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c526f59-08a9-41f4-a451-bda025c0d7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:59.402257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c526f59-08a9-41f4-a451-bda025c0d7ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:59.402543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c526f59-08a9-41f4-a451-bda025c0d7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:59.402820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c526f59-08a9-41f4-a451-bda025c0d7ef admin admin] PATH_INFO: `/v3/projects/35c5594ff0d34b35af8139dc98ed1c5d/users/9a6159de6d62496ca664579e6a1d7d7a/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:59.404854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c526f59-08a9-41f4-a451-bda025c0d7ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c5594ff0d34b35af8139dc98ed1c5d, user_id=9a6159de6d62496ca664579e6a1d7d7a, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:59.407519 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c526f59-08a9-41f4-a451-bda025c0d7ef admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:59.424082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1218/2440] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:47:59 2022] PUT /identity/v3/projects/35c5594ff0d34b35af8139dc98ed1c5d/users/9a6159de6d62496ca664579e6a1d7d7a/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:47:59.431059 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29df2d-73d8-4dd9-94d0-f37d2b092039 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:59.431488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29df2d-73d8-4dd9-94d0-f37d2b092039 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:59.431777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29df2d-73d8-4dd9-94d0-f37d2b092039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:59.472156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0f29df2d-73d8-4dd9-94d0-f37d2b092039 None None] MFA Rules not processed for user `9a6159de6d62496ca664579e6a1d7d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:59.474050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0f29df2d-73d8-4dd9-94d0-f37d2b092039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:59.483626 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:59.496822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:47:59.498367 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f842a3f-f246-49d8-b16b-094dc54ef01b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c2b47da3e174daebf5332ebf61026fd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:59.499337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f842a3f-f246-49d8-b16b-094dc54ef01b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c2b47da3e174daebf5332ebf61026fd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:59.509011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f842a3f-f246-49d8-b16b-094dc54ef01b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c2b47da3e174daebf5332ebf61026fd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:59.523678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f842a3f-f246-49d8-b16b-094dc54ef01b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c2b47da3e174daebf5332ebf61026fd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:59.577858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1219/2441] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:59 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:59.590844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29df2d-73d8-4dd9-94d0-f37d2b092039 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:59.592325 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0f29df2d-73d8-4dd9-94d0-f37d2b092039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:59.620417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f842a3f-f246-49d8-b16b-094dc54ef01b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c2b47da3e174daebf5332ebf61026fd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:47:59.630164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1223/2442] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:47:59 2022] DELETE /identity/v3/users/2c2b47da3e174daebf5332ebf61026fd => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:47:59.631945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48c6f2-568f-4ce5-b6db-086640662141 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:59.632473 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48c6f2-568f-4ce5-b6db-086640662141 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:59.632782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48c6f2-568f-4ce5-b6db-086640662141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:59.658259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4b48c6f2-568f-4ce5-b6db-086640662141 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:47:59.660462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4b48c6f2-568f-4ce5-b6db-086640662141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:59.665163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29df2d-73d8-4dd9-94d0-f37d2b092039 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:59.724012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ccc54-ee53-42ef-99a0-22e17c948dd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:59.724576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ccc54-ee53-42ef-99a0-22e17c948dd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:59.724855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ccc54-ee53-42ef-99a0-22e17c948dd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:59.725169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ccc54-ee53-42ef-99a0-22e17c948dd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:59.740768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1224/2443] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:47:59.750842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48c6f2-568f-4ce5-b6db-086640662141 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:59.753134 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4b48c6f2-568f-4ce5-b6db-086640662141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:59.813058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ccc54-ee53-42ef-99a0-22e17c948dd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:59.815429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ccc54-ee53-42ef-99a0-22e17c948dd0 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:59.828315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1220/2444] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:59 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:47:59.837730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48c6f2-568f-4ce5-b6db-086640662141 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:59.842796 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb21b9a-ece0-4f93-ac23-dd139dbe9689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:59.843741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21b9a-ece0-4f93-ac23-dd139dbe9689 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:59.844054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21b9a-ece0-4f93-ac23-dd139dbe9689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:59.844343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21b9a-ece0-4f93-ac23-dd139dbe9689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:59.859440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb21b9a-ece0-4f93-ac23-dd139dbe9689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:47:59.860918 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb21b9a-ece0-4f93-ac23-dd139dbe9689 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:47:59.872180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1225/2445] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:47:59.910698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ccc54-ee53-42ef-99a0-22e17c948dd0 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:47:59.912208 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6f5ccc54-ee53-42ef-99a0-22e17c948dd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:59.925290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ccc54-ee53-42ef-99a0-22e17c948dd0 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:47:59.929553 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f70aff-85b0-4958-ac95-76b06632ba21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:47:59.930193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f70aff-85b0-4958-ac95-76b06632ba21 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:47:59.930843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f70aff-85b0-4958-ac95-76b06632ba21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:47:59.931128 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f70aff-85b0-4958-ac95-76b06632ba21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:47:59.938773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a7f70aff-85b0-4958-ac95-76b06632ba21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:47:59.962375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:00.035861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f70aff-85b0-4958-ac95-76b06632ba21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:00.037926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f70aff-85b0-4958-ac95-76b06632ba21 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:00.060332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1221/2446] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:47:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:00.447115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb21b9a-ece0-4f93-ac23-dd139dbe9689 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:00.447115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eeb21b9a-ece0-4f93-ac23-dd139dbe9689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:00.463106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb21b9a-ece0-4f93-ac23-dd139dbe9689 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:00.468008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:00.468803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:00.469154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:00.469520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] PATH_INFO: `/v3/projects/8548962986344cf5a4b34ab004e5ce88` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:00.479040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8548962986344cf5a4b34ab004e5ce88)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:00.485695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:00.640643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8548962986344cf5a4b34ab004e5ce88', 'request_id': 'req-ba0612c8-ee0e-4b2f-a434-c42635538722'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:00.785243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8548962986344cf5a4b34ab004e5ce88', 'request_id': 'req-ba0612c8-ee0e-4b2f-a434-c42635538722'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:00.786763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:00.799977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8548962986344cf5a4b34ab004e5ce88', 'request_id': 'req-ba0612c8-ee0e-4b2f-a434-c42635538722'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:00.834436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] The token cache is being invalidate because project 8548962986344cf5a4b34ab004e5ce88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:48:00.835090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:00.836213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba0612c8-ee0e-4b2f-a434-c42635538722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:00.838188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1226/2447] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:48:00 2022] DELETE /identity/v3/projects/8548962986344cf5a4b34ab004e5ce88 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:00.843669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb726f7d-06cc-4012-9db2-0d76b2fbf170 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:00.844267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb726f7d-06cc-4012-9db2-0d76b2fbf170 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:00.844267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb726f7d-06cc-4012-9db2-0d76b2fbf170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:00.867618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fb726f7d-06cc-4012-9db2-0d76b2fbf170 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:00.868874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fb726f7d-06cc-4012-9db2-0d76b2fbf170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:00.931608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1222/2448] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:01.032762 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db418ee-648b-463e-80a0-250add6c77b1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:01.033131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db418ee-648b-463e-80a0-250add6c77b1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:01.033427 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db418ee-648b-463e-80a0-250add6c77b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:01.055444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0db418ee-648b-463e-80a0-250add6c77b1 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:01.056715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0db418ee-648b-463e-80a0-250add6c77b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:01.064782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1227/2449] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:48:01.069635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb726f7d-06cc-4012-9db2-0d76b2fbf170 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:01.079609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb726f7d-06cc-4012-9db2-0d76b2fbf170 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:01.084323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045d0a5-b926-4b46-bd65-59f0f3d3faed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:01.084936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045d0a5-b926-4b46-bd65-59f0f3d3faed admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:01.085229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045d0a5-b926-4b46-bd65-59f0f3d3faed admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:01.085511 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045d0a5-b926-4b46-bd65-59f0f3d3faed admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:01.086202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045d0a5-b926-4b46-bd65-59f0f3d3faed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:48:01.086509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045d0a5-b926-4b46-bd65-59f0f3d3faed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:01.088140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045d0a5-b926-4b46-bd65-59f0f3d3faed admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:01.105960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1223/2450] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:48:01.107058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db418ee-648b-463e-80a0-250add6c77b1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:01.117312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db418ee-648b-463e-80a0-250add6c77b1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:01.122126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b114da2-337d-4a19-a08f-1e0c092216e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:01.122742 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b114da2-337d-4a19-a08f-1e0c092216e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:01.123028 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b114da2-337d-4a19-a08f-1e0c092216e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:01.123300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b114da2-337d-4a19-a08f-1e0c092216e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:01.123971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b114da2-337d-4a19-a08f-1e0c092216e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:01.125939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b114da2-337d-4a19-a08f-1e0c092216e3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:01.277769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1228/2451] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:48:01 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:01.289330 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045d0a5-b926-4b46-bd65-59f0f3d3faed admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:01.300660 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045d0a5-b926-4b46-bd65-59f0f3d3faed admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:01.358390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f7e4b-3d57-44c1-a8c2-f88c7a308647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:01.359021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f7e4b-3d57-44c1-a8c2-f88c7a308647 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:01.359295 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f7e4b-3d57-44c1-a8c2-f88c7a308647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:01.359575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f7e4b-3d57-44c1-a8c2-f88c7a308647 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:01.360279 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f7e4b-3d57-44c1-a8c2-f88c7a308647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:01.362201 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f7e4b-3d57-44c1-a8c2-f88c7a308647 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:01.480472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1224/2452] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:48:01 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:01.484558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b114da2-337d-4a19-a08f-1e0c092216e3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:01.495517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b114da2-337d-4a19-a08f-1e0c092216e3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:01.499999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ec6e8-0485-474f-a199-976ceee1ffb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:01.500658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ec6e8-0485-474f-a199-976ceee1ffb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:01.500938 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ec6e8-0485-474f-a199-976ceee1ffb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:01.501215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ec6e8-0485-474f-a199-976ceee1ffb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:01.501907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ec6e8-0485-474f-a199-976ceee1ffb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:01.503520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ec6e8-0485-474f-a199-976ceee1ffb0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:01.515582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1229/2453] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:01.520126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f7e4b-3d57-44c1-a8c2-f88c7a308647 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:01.531617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f7e4b-3d57-44c1-a8c2-f88c7a308647 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:01.538089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26bdf1-4c10-4c6d-a825-e55664a78cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:01.538748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26bdf1-4c10-4c6d-a825-e55664a78cff admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:01.539037 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26bdf1-4c10-4c6d-a825-e55664a78cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:01.539469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26bdf1-4c10-4c6d-a825-e55664a78cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:01.540254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26bdf1-4c10-4c6d-a825-e55664a78cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:01.542041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26bdf1-4c10-4c6d-a825-e55664a78cff admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:01.553558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1225/2454] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:01.557832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ec6e8-0485-474f-a199-976ceee1ffb0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:01.567602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ec6e8-0485-474f-a199-976ceee1ffb0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:01.573561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94359a09-447e-4f9a-8bb2-0ee27089c5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:01.574785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94359a09-447e-4f9a-8bb2-0ee27089c5d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:01.574785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94359a09-447e-4f9a-8bb2-0ee27089c5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:01.574785 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94359a09-447e-4f9a-8bb2-0ee27089c5d9 admin admin] PATH_INFO: `/v3/projects/78ce0150832b415abb997c2e26e5a007/users/2979498fe4614879800a313e1a932fc3/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:01.591647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94359a09-447e-4f9a-8bb2-0ee27089c5d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ce0150832b415abb997c2e26e5a007, user_id=2979498fe4614879800a313e1a932fc3, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:01.594389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94359a09-447e-4f9a-8bb2-0ee27089c5d9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:01.750582 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1230/2455] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:01 2022] PUT /identity/v3/projects/78ce0150832b415abb997c2e26e5a007/users/2979498fe4614879800a313e1a932fc3/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:01.755780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c750afd-9a0c-4991-8a23-c60eae198622 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:01.756103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c750afd-9a0c-4991-8a23-c60eae198622 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:01.756439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c750afd-9a0c-4991-8a23-c60eae198622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:01.779895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9c750afd-9a0c-4991-8a23-c60eae198622 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:01.781174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9c750afd-9a0c-4991-8a23-c60eae198622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:01.796404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:01.874091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1226/2456] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:01.884863 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94359a09-447e-4f9a-8bb2-0ee27089c5d9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:01.886583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-94359a09-447e-4f9a-8bb2-0ee27089c5d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:01.960081 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94359a09-447e-4f9a-8bb2-0ee27089c5d9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:01.964380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35181c32-fb2b-43f9-9d0f-f0e23c9478f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:01.965018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35181c32-fb2b-43f9-9d0f-f0e23c9478f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:01.965299 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35181c32-fb2b-43f9-9d0f-f0e23c9478f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:01.965573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35181c32-fb2b-43f9-9d0f-f0e23c9478f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:01.976448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35181c32-fb2b-43f9-9d0f-f0e23c9478f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:01.977572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35181c32-fb2b-43f9-9d0f-f0e23c9478f8 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:01.990177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1231/2457] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:48:06.208466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c750afd-9a0c-4991-8a23-c60eae198622 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:06.211047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9c750afd-9a0c-4991-8a23-c60eae198622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:06.225728 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c750afd-9a0c-4991-8a23-c60eae198622 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:06.236517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:06.237145 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:06.237596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:06.238543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] PATH_INFO: `/v3/users/b24da326dc0e4e52ae9c0c5825575f60` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:06.240458 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b24da326dc0e4e52ae9c0c5825575f60)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:06.244425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:06.510170 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b24da326dc0e4e52ae9c0c5825575f60'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:06.511107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b24da326dc0e4e52ae9c0c5825575f60'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:06.511941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b24da326dc0e4e52ae9c0c5825575f60'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:06.526632 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b24da326dc0e4e52ae9c0c5825575f60'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:06.610869 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b24da326dc0e4e52ae9c0c5825575f60'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:06.631306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1227/2458] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:48:06 2022] DELETE /identity/v3/users/b24da326dc0e4e52ae9c0c5825575f60 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:06.635597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e5517-0025-4975-8af4-a3886f803712 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:06.637592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e5517-0025-4975-8af4-a3886f803712 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:06.637592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e5517-0025-4975-8af4-a3886f803712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:06.661484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-656e5517-0025-4975-8af4-a3886f803712 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:06.663419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-656e5517-0025-4975-8af4-a3886f803712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:06.745979 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1232/2459] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:06.759711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:06.761621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:06.874955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79471ba0-d3f6-4981-8e06-64d8732b1b72 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:06.891735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d30f7-9b36-4245-bbf9-5ea605821c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:06.893471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d30f7-9b36-4245-bbf9-5ea605821c21 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:06.893768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d30f7-9b36-4245-bbf9-5ea605821c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:06.894048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d30f7-9b36-4245-bbf9-5ea605821c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:06.911817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d30f7-9b36-4245-bbf9-5ea605821c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:06.913497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d30f7-9b36-4245-bbf9-5ea605821c21 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:06.926152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1228/2460] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:07.753598 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e5517-0025-4975-8af4-a3886f803712 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:07.761133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-656e5517-0025-4975-8af4-a3886f803712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:07.779120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e5517-0025-4975-8af4-a3886f803712 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:07.804063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:07.805350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:07.805808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:07.806399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] PATH_INFO: `/v3/projects/27f2406f6d8d45a0adba69c10261478e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:07.807904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f2406f6d8d45a0adba69c10261478e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:07.810754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:07.845649 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:08.042865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f2406f6d8d45a0adba69c10261478e', 'request_id': 'req-386d3bc3-beb1-4163-bec0-cad076ca2a7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:08.136955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f2406f6d8d45a0adba69c10261478e', 'request_id': 'req-386d3bc3-beb1-4163-bec0-cad076ca2a7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:08.149000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f2406f6d8d45a0adba69c10261478e', 'request_id': 'req-386d3bc3-beb1-4163-bec0-cad076ca2a7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:08.173364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] The token cache is being invalidate because project 27f2406f6d8d45a0adba69c10261478e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:48:08.174070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:08.174980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-386d3bc3-beb1-4163-bec0-cad076ca2a7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:08.179184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1233/2461] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:48:07 2022] DELETE /identity/v3/projects/27f2406f6d8d45a0adba69c10261478e => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:08.182981 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d30f7-9b36-4245-bbf9-5ea605821c21 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:08.195849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d30f7-9b36-4245-bbf9-5ea605821c21 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:08.235484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:08.260194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:08.261049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:08.261532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:08.261815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] PATH_INFO: `/v3/users/9a6159de6d62496ca664579e6a1d7d7a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:08.263000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6159de6d62496ca664579e6a1d7d7a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:08.265360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:08.508569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6159de6d62496ca664579e6a1d7d7a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:08.509407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6159de6d62496ca664579e6a1d7d7a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:08.510052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6159de6d62496ca664579e6a1d7d7a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:08.524035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6159de6d62496ca664579e6a1d7d7a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:08.610926 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6159de6d62496ca664579e6a1d7d7a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:08.637231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1229/2462] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:48:08 2022] DELETE /identity/v3/users/9a6159de6d62496ca664579e6a1d7d7a => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:48:09.115110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38737084-f88d-46de-9f5e-7e610e3f0335 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.115641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38737084-f88d-46de-9f5e-7e610e3f0335 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.115641 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38737084-f88d-46de-9f5e-7e610e3f0335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.155759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-38737084-f88d-46de-9f5e-7e610e3f0335 None None] MFA Rules not processed for user `2979498fe4614879800a313e1a932fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:09.157110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-38737084-f88d-46de-9f5e-7e610e3f0335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:09.167205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:09.168900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:09.263669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75f95f-5c8c-4f42-a590-cc21e57237cf admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:09.268984 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:09.269698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.269983 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.272074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] PATH_INFO: `/v3/projects/35c5594ff0d34b35af8139dc98ed1c5d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.272074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] RBAC: Authorizing `identity:delete_project(project_id=35c5594ff0d34b35af8139dc98ed1c5d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:09.273671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:09.283808 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1234/2463] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:09.288470 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38737084-f88d-46de-9f5e-7e610e3f0335 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:09.290534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-38737084-f88d-46de-9f5e-7e610e3f0335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:09.304877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38737084-f88d-46de-9f5e-7e610e3f0335 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:09.309991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10e8b5-7173-4bc3-bc3d-4c612884bf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:09.310708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10e8b5-7173-4bc3-bc3d-4c612884bf64 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.311096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10e8b5-7173-4bc3-bc3d-4c612884bf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.311314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10e8b5-7173-4bc3-bc3d-4c612884bf64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.312040 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10e8b5-7173-4bc3-bc3d-4c612884bf64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:09.314047 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10e8b5-7173-4bc3-bc3d-4c612884bf64 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:09.421049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c5594ff0d34b35af8139dc98ed1c5d', 'request_id': 'req-aa4da4f2-2178-480e-8f92-db507cc4b31f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:09.427034 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c5594ff0d34b35af8139dc98ed1c5d', 'request_id': 'req-aa4da4f2-2178-480e-8f92-db507cc4b31f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:09.466384 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1235/2464] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:48:09 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:48:09.467294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10e8b5-7173-4bc3-bc3d-4c612884bf64 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:09.531828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10e8b5-7173-4bc3-bc3d-4c612884bf64 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:09.531975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c5594ff0d34b35af8139dc98ed1c5d', 'request_id': 'req-aa4da4f2-2178-480e-8f92-db507cc4b31f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:09.537335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380b9ab-48c1-425b-ad67-2ead984b6330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:09.538757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b9ab-48c1-425b-ad67-2ead984b6330 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.539232 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b9ab-48c1-425b-ad67-2ead984b6330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.539664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b9ab-48c1-425b-ad67-2ead984b6330 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.540546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380b9ab-48c1-425b-ad67-2ead984b6330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:09.542696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380b9ab-48c1-425b-ad67-2ead984b6330 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:09.549503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] The token cache is being invalidate because project 35c5594ff0d34b35af8139dc98ed1c5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:48:09.549894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:09.552670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-aa4da4f2-2178-480e-8f92-db507cc4b31f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:09.559787 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1230/2465] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:48:09 2022] DELETE /identity/v3/projects/35c5594ff0d34b35af8139dc98ed1c5d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:09.563699 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9852c-28c3-4f74-97e2-8034dccaf1dc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.564085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9852c-28c3-4f74-97e2-8034dccaf1dc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.564575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9852c-28c3-4f74-97e2-8034dccaf1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.595020 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bdd9852c-28c3-4f74-97e2-8034dccaf1dc None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:09.596265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bdd9852c-28c3-4f74-97e2-8034dccaf1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:09.605594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1231/2466] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:09.613602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b99c-9166-46ac-972f-21cfbe9dafea None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.613950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b99c-9166-46ac-972f-21cfbe9dafea None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.614236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b99c-9166-46ac-972f-21cfbe9dafea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.634713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1236/2467] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:48:09 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:09.635536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-85d9b99c-9166-46ac-972f-21cfbe9dafea None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:09.636822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-85d9b99c-9166-46ac-972f-21cfbe9dafea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:09.638746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380b9ab-48c1-425b-ad67-2ead984b6330 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:09.641431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a380b9ab-48c1-425b-ad67-2ead984b6330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:09.646712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1232/2468] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:48:09.651116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b99c-9166-46ac-972f-21cfbe9dafea None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:09.654510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380b9ab-48c1-425b-ad67-2ead984b6330 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:09.659787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36217ed-774a-4cd2-bdab-dc38fe241086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:09.660449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36217ed-774a-4cd2-bdab-dc38fe241086 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.660582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b99c-9166-46ac-972f-21cfbe9dafea None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:09.660753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36217ed-774a-4cd2-bdab-dc38fe241086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.661015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36217ed-774a-4cd2-bdab-dc38fe241086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.661813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36217ed-774a-4cd2-bdab-dc38fe241086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:09.663817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36217ed-774a-4cd2-bdab-dc38fe241086 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:09.665395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b087ff9-430c-4a2e-8903-5ca0699938c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:09.666017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087ff9-430c-4a2e-8903-5ca0699938c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.666302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087ff9-430c-4a2e-8903-5ca0699938c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.666581 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087ff9-430c-4a2e-8903-5ca0699938c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.667275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b087ff9-430c-4a2e-8903-5ca0699938c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:48:09.667590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b087ff9-430c-4a2e-8903-5ca0699938c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:09.669268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b087ff9-430c-4a2e-8903-5ca0699938c6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:09.674163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1237/2469] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:48:09.678896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36217ed-774a-4cd2-bdab-dc38fe241086 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:09.681888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1233/2470] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:48:09.688659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36217ed-774a-4cd2-bdab-dc38fe241086 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:09.690176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b087ff9-430c-4a2e-8903-5ca0699938c6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:09.693485 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2d25f-e703-41e1-b654-7bd635377a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:09.694804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2d25f-e703-41e1-b654-7bd635377a67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.695086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2d25f-e703-41e1-b654-7bd635377a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.695363 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2d25f-e703-41e1-b654-7bd635377a67 admin admin] PATH_INFO: `/v3/projects/1ac52420449143a39f8f377499804dc5/users/c5d08b60a0684fc180b17b37015c3865/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.700571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b087ff9-430c-4a2e-8903-5ca0699938c6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:09.705588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ce01a-ecb5-46b7-851a-95e042b9da53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:09.706182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce01a-ecb5-46b7-851a-95e042b9da53 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.706466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce01a-ecb5-46b7-851a-95e042b9da53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.706745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce01a-ecb5-46b7-851a-95e042b9da53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.707411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ce01a-ecb5-46b7-851a-95e042b9da53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:09.709386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ce01a-ecb5-46b7-851a-95e042b9da53 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:09.710599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2d25f-e703-41e1-b654-7bd635377a67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac52420449143a39f8f377499804dc5, user_id=c5d08b60a0684fc180b17b37015c3865, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:09.713330 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2d25f-e703-41e1-b654-7bd635377a67 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:09.788035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1238/2471] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:09 2022] PUT /identity/v3/projects/1ac52420449143a39f8f377499804dc5/users/c5d08b60a0684fc180b17b37015c3865/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:09.791464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2d25f-e703-41e1-b654-7bd635377a67 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:09.846597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:09.856056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1234/2472] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:48:09 2022] POST /identity/v3/projects => generated 401 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:48:09.857355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ce01a-ecb5-46b7-851a-95e042b9da53 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:09.857891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2d25f-e703-41e1-b654-7bd635377a67 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:09.921819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e836dab-22f3-4e5c-95f6-104c99d3c2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:09.922528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e836dab-22f3-4e5c-95f6-104c99d3c2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.922885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e836dab-22f3-4e5c-95f6-104c99d3c2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.923112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e836dab-22f3-4e5c-95f6-104c99d3c2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.923821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e836dab-22f3-4e5c-95f6-104c99d3c2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:09.925320 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ce01a-ecb5-46b7-851a-95e042b9da53 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:09.927904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e836dab-22f3-4e5c-95f6-104c99d3c2d5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:09.930703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a21c8-fe7d-4ea4-a712-b2b8276440d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:09.931322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a21c8-fe7d-4ea4-a712-b2b8276440d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.931606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a21c8-fe7d-4ea4-a712-b2b8276440d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.931883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a21c8-fe7d-4ea4-a712-b2b8276440d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.932590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a21c8-fe7d-4ea4-a712-b2b8276440d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:09.934500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a21c8-fe7d-4ea4-a712-b2b8276440d9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:09.935833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1239/2473] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:48:09.939873 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e836dab-22f3-4e5c-95f6-104c99d3c2d5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:09.950336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e836dab-22f3-4e5c-95f6-104c99d3c2d5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:09.960214 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79694de-d0c4-4e21-ba67-f090ffa6081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:09.960907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79694de-d0c4-4e21-ba67-f090ffa6081d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:09.963309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79694de-d0c4-4e21-ba67-f090ffa6081d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:09.963309 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79694de-d0c4-4e21-ba67-f090ffa6081d admin admin] PATH_INFO: `/v3/projects/1ac52420449143a39f8f377499804dc5/users/c5d08b60a0684fc180b17b37015c3865/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:09.965441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79694de-d0c4-4e21-ba67-f090ffa6081d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac52420449143a39f8f377499804dc5, user_id=c5d08b60a0684fc180b17b37015c3865, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:09.968196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79694de-d0c4-4e21-ba67-f090ffa6081d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:09.979452 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1240/2474] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:09 2022] PUT /identity/v3/projects/1ac52420449143a39f8f377499804dc5/users/c5d08b60a0684fc180b17b37015c3865/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:48:10.043206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1235/2475] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:48:09 2022] POST /identity/v3/users => generated 301 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:48:10.047174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79694de-d0c4-4e21-ba67-f090ffa6081d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:10.117640 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79694de-d0c4-4e21-ba67-f090ffa6081d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:10.122421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0462831-8d90-4a75-bd3c-df74595d93b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:10.123320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0462831-8d90-4a75-bd3c-df74595d93b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:10.123607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0462831-8d90-4a75-bd3c-df74595d93b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:10.124274 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0462831-8d90-4a75-bd3c-df74595d93b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:10.134128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0462831-8d90-4a75-bd3c-df74595d93b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:10.134128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0462831-8d90-4a75-bd3c-df74595d93b2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:10.138443 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1241/2476] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:10.142520 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a21c8-fe7d-4ea4-a712-b2b8276440d9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:10.165056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:10.173859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a21c8-fe7d-4ea4-a712-b2b8276440d9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:10.187646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a696799-2724-4fa6-93de-b7173e7577c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:10.188325 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a696799-2724-4fa6-93de-b7173e7577c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:10.188614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a696799-2724-4fa6-93de-b7173e7577c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:10.188892 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a696799-2724-4fa6-93de-b7173e7577c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:10.189605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a696799-2724-4fa6-93de-b7173e7577c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:10.191258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a696799-2724-4fa6-93de-b7173e7577c3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:10.214643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1236/2477] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:48:10.219005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0462831-8d90-4a75-bd3c-df74595d93b2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:10.230754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0462831-8d90-4a75-bd3c-df74595d93b2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:10.238325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2029c97-88a0-48fc-ae9f-f1c9dfdd6077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:10.239050 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029c97-88a0-48fc-ae9f-f1c9dfdd6077 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:10.239335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029c97-88a0-48fc-ae9f-f1c9dfdd6077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:10.239614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029c97-88a0-48fc-ae9f-f1c9dfdd6077 admin admin] PATH_INFO: `/v3/projects/13cb49c633494e78855dc4de4a05e400/users/91512b6ceb324e3489a2dcf965f5a217/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:10.257192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2029c97-88a0-48fc-ae9f-f1c9dfdd6077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13cb49c633494e78855dc4de4a05e400, user_id=91512b6ceb324e3489a2dcf965f5a217, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:10.260023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2029c97-88a0-48fc-ae9f-f1c9dfdd6077 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:10.349747 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1242/2478] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:10 2022] PUT /identity/v3/projects/13cb49c633494e78855dc4de4a05e400/users/91512b6ceb324e3489a2dcf965f5a217/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:48:10.362586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b2f4-683f-40ac-be5b-ad6ed299fd1d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:10.363033 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b2f4-683f-40ac-be5b-ad6ed299fd1d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:10.363113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b2f4-683f-40ac-be5b-ad6ed299fd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:10.393502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8193b2f4-683f-40ac-be5b-ad6ed299fd1d None None] MFA Rules not processed for user `91512b6ceb324e3489a2dcf965f5a217`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:10.394797 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8193b2f4-683f-40ac-be5b-ad6ed299fd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:10.486266 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1237/2479] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:10 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:10.487625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2029c97-88a0-48fc-ae9f-f1c9dfdd6077 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:10.566856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2029c97-88a0-48fc-ae9f-f1c9dfdd6077 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:10.572948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd4b27-ce8a-42bf-9252-5cb1e605bf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:10.574162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd4b27-ce8a-42bf-9252-5cb1e605bf0b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:10.574538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd4b27-ce8a-42bf-9252-5cb1e605bf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:10.575618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd4b27-ce8a-42bf-9252-5cb1e605bf0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:10.576453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd4b27-ce8a-42bf-9252-5cb1e605bf0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:10.579558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd4b27-ce8a-42bf-9252-5cb1e605bf0b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:10.593112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5489a17-dfc4-414b-8566-43c0c98c9a57 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:10.593870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5489a17-dfc4-414b-8566-43c0c98c9a57 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:10.594429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5489a17-dfc4-414b-8566-43c0c98c9a57 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:10.596440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1238/2480] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:48:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:48:10.603258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f010f51-eea8-4f49-9b9c-d5ca811e4ab7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:10.608900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f010f51-eea8-4f49-9b9c-d5ca811e4ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:10.608900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f010f51-eea8-4f49-9b9c-d5ca811e4ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:10.639890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2f010f51-eea8-4f49-9b9c-d5ca811e4ab7 None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:10.643373 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2f010f51-eea8-4f49-9b9c-d5ca811e4ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:10.701396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1243/2481] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:48:10 2022] POST /identity/v3/projects => generated 399 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:10.718983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd4b27-ce8a-42bf-9252-5cb1e605bf0b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:10.763728 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1239/2482] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:48:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:48:10.785760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f010f51-eea8-4f49-9b9c-d5ca811e4ab7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:10.785760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2f010f51-eea8-4f49-9b9c-d5ca811e4ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:10.796535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd4b27-ce8a-42bf-9252-5cb1e605bf0b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:10.801287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb576a-c41b-479d-b198-0ed5dcd34d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:10.802008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb576a-c41b-479d-b198-0ed5dcd34d54 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:10.802354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb576a-c41b-479d-b198-0ed5dcd34d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:10.802691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb576a-c41b-479d-b198-0ed5dcd34d54 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:10.803463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb576a-c41b-479d-b198-0ed5dcd34d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:10.820533 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb576a-c41b-479d-b198-0ed5dcd34d54 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:10.868035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f010f51-eea8-4f49-9b9c-d5ca811e4ab7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:10.941183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1244/2483] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:48:10 2022] POST /identity/v3/users => generated 306 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:10.945879 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb576a-c41b-479d-b198-0ed5dcd34d54 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:10.964411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454a8d0-e21e-4b99-93cc-0a29f40cb491 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfc4dfda424d40b7c9f17345b8ad45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:10.965298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454a8d0-e21e-4b99-93cc-0a29f40cb491 service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:10.966371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454a8d0-e21e-4b99-93cc-0a29f40cb491 service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:10.966654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454a8d0-e21e-4b99-93cc-0a29f40cb491 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:10.973592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb576a-c41b-479d-b198-0ed5dcd34d54 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:10.986286 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec9d6-7337-4606-8b80-8202580001db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:10.987096 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec9d6-7337-4606-8b80-8202580001db admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:10.987388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec9d6-7337-4606-8b80-8202580001db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:10.987666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec9d6-7337-4606-8b80-8202580001db admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:10.989706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ec9d6-7337-4606-8b80-8202580001db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:10.994537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ec9d6-7337-4606-8b80-8202580001db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:11.019577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1245/2484] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:11.024229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec9d6-7337-4606-8b80-8202580001db admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:11.042097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec9d6-7337-4606-8b80-8202580001db admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:11.052239 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93f24b-265a-42b0-b67d-ccc77178d8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:11.053944 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f24b-265a-42b0-b67d-ccc77178d8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.054359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f24b-265a-42b0-b67d-ccc77178d8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.059517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f24b-265a-42b0-b67d-ccc77178d8b6 admin admin] PATH_INFO: `/v3/projects/769e435f0f074ac88999d82674550493/users/370665c2ebaa4bfcb212c59849c05abe/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.067514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454a8d0-e21e-4b99-93cc-0a29f40cb491 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:11.069772 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454a8d0-e21e-4b99-93cc-0a29f40cb491 service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:11.076869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93f24b-265a-42b0-b67d-ccc77178d8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=769e435f0f074ac88999d82674550493, user_id=370665c2ebaa4bfcb212c59849c05abe, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:11.082546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93f24b-265a-42b0-b67d-ccc77178d8b6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:11.101149 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1240/2485] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:48:11.102092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454a8d0-e21e-4b99-93cc-0a29f40cb491 service glance] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:11.114312 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454a8d0-e21e-4b99-93cc-0a29f40cb491 service glance] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:11.120116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8556c-ac4f-4e44-a6a2-49fe6c5f75eb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfc4dfda424d40b7c9f17345b8ad45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:11.120747 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8556c-ac4f-4e44-a6a2-49fe6c5f75eb service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.121689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8556c-ac4f-4e44-a6a2-49fe6c5f75eb service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.121967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8556c-ac4f-4e44-a6a2-49fe6c5f75eb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.123961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-23a8556c-ac4f-4e44-a6a2-49fe6c5f75eb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:11.173287 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1246/2486] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:11 2022] PUT /identity/v3/projects/769e435f0f074ac88999d82674550493/users/370665c2ebaa4bfcb212c59849c05abe/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:48:11.177756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93f24b-265a-42b0-b67d-ccc77178d8b6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:11.197932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8556c-ac4f-4e44-a6a2-49fe6c5f75eb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:11.200383 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8556c-ac4f-4e44-a6a2-49fe6c5f75eb service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:11.212348 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1241/2487] 10.222.0.64 () {60 vars in 1396 bytes} [Thu Aug 4 07:48:11 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:11.261663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93f24b-265a-42b0-b67d-ccc77178d8b6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:11.270328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5039e60-7717-428b-8a65-136bf5bd4d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:11.270959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039e60-7717-428b-8a65-136bf5bd4d2e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.271410 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039e60-7717-428b-8a65-136bf5bd4d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.272530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039e60-7717-428b-8a65-136bf5bd4d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.273428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5039e60-7717-428b-8a65-136bf5bd4d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:11.275240 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5039e60-7717-428b-8a65-136bf5bd4d2e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:11.290985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1247/2488] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:48:11.295061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8556c-ac4f-4e44-a6a2-49fe6c5f75eb service glance] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:11.306439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8556c-ac4f-4e44-a6a2-49fe6c5f75eb service glance] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:11.312181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef184d-5fca-4f44-b9e2-9fb17a416ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:11.312799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef184d-5fca-4f44-b9e2-9fb17a416ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.313078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef184d-5fca-4f44-b9e2-9fb17a416ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.313351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef184d-5fca-4f44-b9e2-9fb17a416ba9 admin admin] PATH_INFO: `/v3/projects/769e435f0f074ac88999d82674550493/users/370665c2ebaa4bfcb212c59849c05abe/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.315342 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef184d-5fca-4f44-b9e2-9fb17a416ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=769e435f0f074ac88999d82674550493, user_id=370665c2ebaa4bfcb212c59849c05abe, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:11.318068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef184d-5fca-4f44-b9e2-9fb17a416ba9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:11.330706 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1242/2489] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:11 2022] PUT /identity/v3/projects/769e435f0f074ac88999d82674550493/users/370665c2ebaa4bfcb212c59849c05abe/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:48:11.341691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797f3db-683f-4160-a9b3-42f4f7e92b26 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.341994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797f3db-683f-4160-a9b3-42f4f7e92b26 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.342275 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797f3db-683f-4160-a9b3-42f4f7e92b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.380849 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7797f3db-683f-4160-a9b3-42f4f7e92b26 None None] MFA Rules not processed for user `370665c2ebaa4bfcb212c59849c05abe`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:11.382753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7797f3db-683f-4160-a9b3-42f4f7e92b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:11.458647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1248/2490] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:11 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:11.470490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef184d-5fca-4f44-b9e2-9fb17a416ba9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:11.477285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e5ef184d-5fca-4f44-b9e2-9fb17a416ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:11.484657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a955c7-648c-4917-b760-e17a62a94206 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.485034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a955c7-648c-4917-b760-e17a62a94206 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.485400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a955c7-648c-4917-b760-e17a62a94206 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.487049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1249/2491] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:48:11.492268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84625873-196b-43bc-b140-5118f8bc0395 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.492757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84625873-196b-43bc-b140-5118f8bc0395 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.493357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84625873-196b-43bc-b140-5118f8bc0395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.516594 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-84625873-196b-43bc-b140-5118f8bc0395 None None] MFA Rules not processed for user `b80e701a81cb4fa893e55288559010f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:11.518276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-84625873-196b-43bc-b140-5118f8bc0395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:11.562768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef184d-5fca-4f44-b9e2-9fb17a416ba9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:11.572764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6603dd-94c1-437d-8e5e-d03c1ce6b579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:11.575199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6603dd-94c1-437d-8e5e-d03c1ce6b579 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.575199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6603dd-94c1-437d-8e5e-d03c1ce6b579 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.576210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6603dd-94c1-437d-8e5e-d03c1ce6b579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.580049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1250/2492] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:48:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:11.592929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6603dd-94c1-437d-8e5e-d03c1ce6b579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:11.594176 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84625873-196b-43bc-b140-5118f8bc0395 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:11.595660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-84625873-196b-43bc-b140-5118f8bc0395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:11.596069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6603dd-94c1-437d-8e5e-d03c1ce6b579 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:11.609255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1243/2493] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:11 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:11.610178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84625873-196b-43bc-b140-5118f8bc0395 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:11.614137 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4cfe8-de65-43c8-a762-cdc03bec4efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:11.614761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4cfe8-de65-43c8-a762-cdc03bec4efe service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.615038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4cfe8-de65-43c8-a762-cdc03bec4efe service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.615311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4cfe8-de65-43c8-a762-cdc03bec4efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.624754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4cfe8-de65-43c8-a762-cdc03bec4efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:11.626202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4cfe8-de65-43c8-a762-cdc03bec4efe service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:11.636647 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1251/2494] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:48:11.765133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ab401-3434-48b2-98aa-ed7ce5d7a35f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.765429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ab401-3434-48b2-98aa-ed7ce5d7a35f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.765726 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ab401-3434-48b2-98aa-ed7ce5d7a35f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.767399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1244/2495] 10.222.0.64 () {54 vars in 921 bytes} [Thu Aug 4 07:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 07:48:11.773919 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc7b53-d23e-4b20-9429-214d13737f74 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.774200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc7b53-d23e-4b20-9429-214d13737f74 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.774473 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc7b53-d23e-4b20-9429-214d13737f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.786326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:11.799616 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-57bc7b53-d23e-4b20-9429-214d13737f74 None None] MFA Rules not processed for user `4eedd711c6114de1bfcdf692ce058794`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:11.800891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-57bc7b53-d23e-4b20-9429-214d13737f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:11.835101 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ab401-3434-48b2-98aa-ed7ce5d7a35f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:11.857274 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ab401-3434-48b2-98aa-ed7ce5d7a35f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:11.865386 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b770126-fc2a-4e2b-8ca5-36293fb2280e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:11.867036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b770126-fc2a-4e2b-8ca5-36293fb2280e service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.867507 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b770126-fc2a-4e2b-8ca5-36293fb2280e service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.868029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b770126-fc2a-4e2b-8ca5-36293fb2280e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.910648 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1252/2496] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Aug 4 07:48:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:11.928420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc7b53-d23e-4b20-9429-214d13737f74 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:11.930440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-57bc7b53-d23e-4b20-9429-214d13737f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:11.946235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc7b53-d23e-4b20-9429-214d13737f74 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:11.960693 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:11.964900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02fb80-589d-4ad1-b531-6ce9f6e7c268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:11.965913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02fb80-589d-4ad1-b531-6ce9f6e7c268 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:11.966397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02fb80-589d-4ad1-b531-6ce9f6e7c268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:11.967301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02fb80-589d-4ad1-b531-6ce9f6e7c268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:11.967611 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b770126-fc2a-4e2b-8ca5-36293fb2280e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:11.969443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b770126-fc2a-4e2b-8ca5-36293fb2280e service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:11.984471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1245/2497] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:11 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:48:11.994587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02fb80-589d-4ad1-b531-6ce9f6e7c268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:11.994587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02fb80-589d-4ad1-b531-6ce9f6e7c268 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:12.011871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1253/2498] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:48:12.521503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b770126-fc2a-4e2b-8ca5-36293fb2280e service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:12.538469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b770126-fc2a-4e2b-8ca5-36293fb2280e service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:12.543454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f91587-1df0-4fab-a15a-71844ab5df23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:12.544524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f91587-1df0-4fab-a15a-71844ab5df23 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:12.545017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f91587-1df0-4fab-a15a-71844ab5df23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:12.545858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f91587-1df0-4fab-a15a-71844ab5df23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:12.548402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b2f91587-1df0-4fab-a15a-71844ab5df23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:12.738582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f91587-1df0-4fab-a15a-71844ab5df23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:12.741749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f91587-1df0-4fab-a15a-71844ab5df23 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:12.761965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1246/2499] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:12.892753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:13.731467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:13.885766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:14.011093 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:14.867756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15831bd7-9582-4f10-a498-e7a3db9353dc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:14.868693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15831bd7-9582-4f10-a498-e7a3db9353dc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:14.868979 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15831bd7-9582-4f10-a498-e7a3db9353dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:14.892030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-15831bd7-9582-4f10-a498-e7a3db9353dc None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:14.893504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-15831bd7-9582-4f10-a498-e7a3db9353dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:14.975844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1254/2500] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:48:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:14.985268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f91587-1df0-4fab-a15a-71844ab5df23 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:14.989399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b2f91587-1df0-4fab-a15a-71844ab5df23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:15.075024 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f91587-1df0-4fab-a15a-71844ab5df23 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:15.080701 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ba22c-580c-47e9-af05-8ad761c76a33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:15.081495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba22c-580c-47e9-af05-8ad761c76a33 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:15.081886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba22c-580c-47e9-af05-8ad761c76a33 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:15.082264 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba22c-580c-47e9-af05-8ad761c76a33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:15.094583 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ba22c-580c-47e9-af05-8ad761c76a33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:15.096935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ba22c-580c-47e9-af05-8ad761c76a33 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:15.109614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1247/2501] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:48:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 07:48:18.097449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15831bd7-9582-4f10-a498-e7a3db9353dc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:18.118694 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15831bd7-9582-4f10-a498-e7a3db9353dc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:18.127064 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:18.127736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:18.128097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:18.128379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] PATH_INFO: `/v3/users/91512b6ceb324e3489a2dcf965f5a217` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:18.129579 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=91512b6ceb324e3489a2dcf965f5a217)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:18.138595 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:18.406102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91512b6ceb324e3489a2dcf965f5a217'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:18.406736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91512b6ceb324e3489a2dcf965f5a217'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:18.418436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91512b6ceb324e3489a2dcf965f5a217'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:18.433757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91512b6ceb324e3489a2dcf965f5a217'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:18.523495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91512b6ceb324e3489a2dcf965f5a217'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:18.535790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1255/2502] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:48:18 2022] DELETE /identity/v3/users/91512b6ceb324e3489a2dcf965f5a217 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:18.538278 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5e9ff-0291-4d0d-8a2e-089cb500507c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:18.538748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5e9ff-0291-4d0d-8a2e-089cb500507c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:18.538832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5e9ff-0291-4d0d-8a2e-089cb500507c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:18.582965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f0d5e9ff-0291-4d0d-8a2e-089cb500507c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:18.585140 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f0d5e9ff-0291-4d0d-8a2e-089cb500507c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:18.825991 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1248/2503] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:18.864433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:18.871689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:18.934036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:19.001009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774888e-9c70-4bf1-8273-5504da9865fc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:19.005215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b31b3-a1e3-4861-8d24-06cb3b9a127e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:19.005984 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b31b3-a1e3-4861-8d24-06cb3b9a127e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:19.006548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b31b3-a1e3-4861-8d24-06cb3b9a127e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:19.007302 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b31b3-a1e3-4861-8d24-06cb3b9a127e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:19.023947 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b31b3-a1e3-4861-8d24-06cb3b9a127e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:19.025683 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b31b3-a1e3-4861-8d24-06cb3b9a127e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:19.038430 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1256/2504] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:20.160267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd18c8-3ebf-4c9f-9027-3a4a2db6f704 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:20.161425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd18c8-3ebf-4c9f-9027-3a4a2db6f704 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:20.162006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd18c8-3ebf-4c9f-9027-3a4a2db6f704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:20.227353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b31b3-a1e3-4861-8d24-06cb3b9a127e service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:20.229655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bf7b31b3-a1e3-4861-8d24-06cb3b9a127e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:20.255340 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-47cd18c8-3ebf-4c9f-9027-3a4a2db6f704 None None] MFA Rules not processed for user `c5d08b60a0684fc180b17b37015c3865`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:20.257214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-47cd18c8-3ebf-4c9f-9027-3a4a2db6f704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:20.266331 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b31b3-a1e3-4861-8d24-06cb3b9a127e service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:20.290042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:20.291244 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:20.291538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:20.291806 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] PATH_INFO: `/v3/projects/13cb49c633494e78855dc4de4a05e400` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:20.293838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13cb49c633494e78855dc4de4a05e400)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:20.295951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:20.457833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13cb49c633494e78855dc4de4a05e400', 'request_id': 'req-4410717e-2de8-4dde-aea7-d3cdbe324aa2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:20.536750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13cb49c633494e78855dc4de4a05e400', 'request_id': 'req-4410717e-2de8-4dde-aea7-d3cdbe324aa2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:20.545233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13cb49c633494e78855dc4de4a05e400', 'request_id': 'req-4410717e-2de8-4dde-aea7-d3cdbe324aa2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:20.561911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] The token cache is being invalidate because project 13cb49c633494e78855dc4de4a05e400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:48:20.562569 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:20.563255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:20.565228 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1257/2505] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:48:20 2022] DELETE /identity/v3/projects/13cb49c633494e78855dc4de4a05e400 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:48:20.575210 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:20.578281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:20.580201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:20.669796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1249/2506] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:20.670960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd18c8-3ebf-4c9f-9027-3a4a2db6f704 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:20.683277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-47cd18c8-3ebf-4c9f-9027-3a4a2db6f704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:20.759700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410717e-2de8-4dde-aea7-d3cdbe324aa2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:20.764710 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f8455-8e80-4347-a82d-badddd247c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:20.765690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8455-8e80-4347-a82d-badddd247c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:20.765690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8455-8e80-4347-a82d-badddd247c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:20.765880 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8455-8e80-4347-a82d-badddd247c9e admin admin] PATH_INFO: `/v3/users/370665c2ebaa4bfcb212c59849c05abe` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:20.767344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f8455-8e80-4347-a82d-badddd247c9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=370665c2ebaa4bfcb212c59849c05abe)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:20.769298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f8455-8e80-4347-a82d-badddd247c9e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:20.805722 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd18c8-3ebf-4c9f-9027-3a4a2db6f704 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:20.909028 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0581f-8c06-4581-9dcc-b944a2caf3b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfc4dfda424d40b7c9f17345b8ad45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:20.909661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0581f-8c06-4581-9dcc-b944a2caf3b6 service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:20.909945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0581f-8c06-4581-9dcc-b944a2caf3b6 service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:20.910222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0581f-8c06-4581-9dcc-b944a2caf3b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:20.912216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-18a0581f-8c06-4581-9dcc-b944a2caf3b6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:21.082269 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0581f-8c06-4581-9dcc-b944a2caf3b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:21.083080 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a25f8455-8e80-4347-a82d-badddd247c9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370665c2ebaa4bfcb212c59849c05abe'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:21.087178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0581f-8c06-4581-9dcc-b944a2caf3b6 service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:21.089728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a25f8455-8e80-4347-a82d-badddd247c9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370665c2ebaa4bfcb212c59849c05abe'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:21.104053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:21.112155 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1250/2507] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:21.112756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a25f8455-8e80-4347-a82d-badddd247c9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370665c2ebaa4bfcb212c59849c05abe'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:21.113112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0581f-8c06-4581-9dcc-b944a2caf3b6 service glance] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:21.117499 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-18a0581f-8c06-4581-9dcc-b944a2caf3b6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:21.135449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a25f8455-8e80-4347-a82d-badddd247c9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370665c2ebaa4bfcb212c59849c05abe'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:21.153202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0581f-8c06-4581-9dcc-b944a2caf3b6 service glance] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:21.163817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:21.166731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:21.166731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:21.166731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] PATH_INFO: `/v3/users/5509519254574e50a70c84ae895acd3f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:21.167283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5509519254574e50a70c84ae895acd3f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:21.186199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:21.221184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a25f8455-8e80-4347-a82d-badddd247c9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370665c2ebaa4bfcb212c59849c05abe'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:21.230379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1258/2508] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:48:20 2022] DELETE /identity/v3/users/370665c2ebaa4bfcb212c59849c05abe => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:21.439770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:21.472828 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5509519254574e50a70c84ae895acd3f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:21.475212 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5509519254574e50a70c84ae895acd3f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:21.476116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5509519254574e50a70c84ae895acd3f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:21.486447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae600d-a983-4b11-b554-d894d4542626 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:21.486707 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae600d-a983-4b11-b554-d894d4542626 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:21.487023 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae600d-a983-4b11-b554-d894d4542626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:21.518463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5509519254574e50a70c84ae895acd3f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:21.529352 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e3ae600d-a983-4b11-b554-d894d4542626 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:21.530844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e3ae600d-a983-4b11-b554-d894d4542626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:21.612692 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5509519254574e50a70c84ae895acd3f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:21.639195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1251/2509] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:48:21 2022] DELETE /identity/v3/users/5509519254574e50a70c84ae895acd3f => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:21.648487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1259/2510] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:48:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:21.674864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:21.676829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:21.807499 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9124f-9e80-4742-a96d-bc72dc6f655b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:21.816282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146c09e-739e-411f-a26b-62a5fc369a7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:21.817792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146c09e-739e-411f-a26b-62a5fc369a7d service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:21.826013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146c09e-739e-411f-a26b-62a5fc369a7d service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:21.827289 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146c09e-739e-411f-a26b-62a5fc369a7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:21.929881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146c09e-739e-411f-a26b-62a5fc369a7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:21.931740 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146c09e-739e-411f-a26b-62a5fc369a7d service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:21.965036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1252/2511] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:48:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:48:22.029140 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae600d-a983-4b11-b554-d894d4542626 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:22.032892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e3ae600d-a983-4b11-b554-d894d4542626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:22.043132 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146c09e-739e-411f-a26b-62a5fc369a7d service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:22.045542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9146c09e-739e-411f-a26b-62a5fc369a7d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:22.277662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:22.314436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146c09e-739e-411f-a26b-62a5fc369a7d service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:22.329790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae600d-a983-4b11-b554-d894d4542626 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:22.354227 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:22.356164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:22.356818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:22.357440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] PATH_INFO: `/v3/projects/fda70ea579f941e6aefa8f06367ab3c9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:22.360369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fda70ea579f941e6aefa8f06367ab3c9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:22.362459 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:22.363675 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:22.367052 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:22.369289 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:22.372443 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] PATH_INFO: `/v3/projects/769e435f0f074ac88999d82674550493` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:22.377280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=769e435f0f074ac88999d82674550493)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:22.384332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:22.606926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fda70ea579f941e6aefa8f06367ab3c9', 'request_id': 'req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:22.617350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '769e435f0f074ac88999d82674550493', 'request_id': 'req-7819a219-e90d-4379-ab0e-ac3c302eb730'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:22.627401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '769e435f0f074ac88999d82674550493', 'request_id': 'req-7819a219-e90d-4379-ab0e-ac3c302eb730'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:22.691558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fda70ea579f941e6aefa8f06367ab3c9', 'request_id': 'req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:22.700751 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fda70ea579f941e6aefa8f06367ab3c9', 'request_id': 'req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:22.722966 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] The token cache is being invalidate because project fda70ea579f941e6aefa8f06367ab3c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:48:22.723276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:22.724336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-032a982d-9f70-4499-bf8b-ad6dfcc65aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:22.727288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1260/2512] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:48:22 2022] DELETE /identity/v3/projects/fda70ea579f941e6aefa8f06367ab3c9 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:22.732146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8123b-c591-459b-bbb0-f50d27be4174 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:22.732512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8123b-c591-459b-bbb0-f50d27be4174 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:22.732858 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8123b-c591-459b-bbb0-f50d27be4174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:22.751392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:22.775125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '769e435f0f074ac88999d82674550493', 'request_id': 'req-7819a219-e90d-4379-ab0e-ac3c302eb730'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:22.793728 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] The token cache is being invalidate because project 769e435f0f074ac88999d82674550493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:48:22.794412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:22.796009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7819a219-e90d-4379-ab0e-ac3c302eb730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:22.797347 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-81f8123b-c591-459b-bbb0-f50d27be4174 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:22.798910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-81f8123b-c591-459b-bbb0-f50d27be4174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:22.799105 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1253/2513] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:48:22 2022] DELETE /identity/v3/projects/769e435f0f074ac88999d82674550493 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:22.804174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b8bb-30fc-48f4-806f-66c4579f3894 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:22.804760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b8bb-30fc-48f4-806f-66c4579f3894 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:22.805190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709b8bb-30fc-48f4-806f-66c4579f3894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:22.828504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e709b8bb-30fc-48f4-806f-66c4579f3894 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:22.830694 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e709b8bb-30fc-48f4-806f-66c4579f3894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:22.918173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1261/2514] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:22.940829 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea58f1-9b95-4ef1-99c7-fb7eb295ae4b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:22.941357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea58f1-9b95-4ef1-99c7-fb7eb295ae4b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:22.947136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea58f1-9b95-4ef1-99c7-fb7eb295ae4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:22.947265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1254/2515] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:48:22.955006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3d5b1-0dd5-4b96-9a34-3a8ef948c6cf None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:22.955517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3d5b1-0dd5-4b96-9a34-3a8ef948c6cf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:22.955928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3d5b1-0dd5-4b96-9a34-3a8ef948c6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:22.984554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fdea58f1-9b95-4ef1-99c7-fb7eb295ae4b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:22.985712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fdea58f1-9b95-4ef1-99c7-fb7eb295ae4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:23.007076 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1262/2516] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:23.008851 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3bb3d5b1-0dd5-4b96-9a34-3a8ef948c6cf None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:23.010319 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3bb3d5b1-0dd5-4b96-9a34-3a8ef948c6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:23.011508 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea58f1-9b95-4ef1-99c7-fb7eb295ae4b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.036475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea58f1-9b95-4ef1-99c7-fb7eb295ae4b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.042714 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1255/2517] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:48:23.048172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3d5b1-0dd5-4b96-9a34-3a8ef948c6cf None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.057477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130809a-e581-4405-89ef-9ac5b8926250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.059546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130809a-e581-4405-89ef-9ac5b8926250 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.060857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130809a-e581-4405-89ef-9ac5b8926250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.062607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130809a-e581-4405-89ef-9ac5b8926250 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.063625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130809a-e581-4405-89ef-9ac5b8926250 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:48:23.067530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130809a-e581-4405-89ef-9ac5b8926250 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.071828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130809a-e581-4405-89ef-9ac5b8926250 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:23.078119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3d5b1-0dd5-4b96-9a34-3a8ef948c6cf None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.097980 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bc810-d3e3-4855-951b-5beaa68a27b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.098669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc810-d3e3-4855-951b-5beaa68a27b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.099018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc810-d3e3-4855-951b-5beaa68a27b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.099332 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc810-d3e3-4855-951b-5beaa68a27b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.100775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bc810-d3e3-4855-951b-5beaa68a27b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:48:23.101410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bc810-d3e3-4855-951b-5beaa68a27b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.106208 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bc810-d3e3-4855-951b-5beaa68a27b6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:23.106734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1263/2518] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:48:23.110813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130809a-e581-4405-89ef-9ac5b8926250 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.138589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130809a-e581-4405-89ef-9ac5b8926250 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.142445 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1256/2519] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:48:23.143563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bc810-d3e3-4855-951b-5beaa68a27b6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.152951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04befd12-ca47-4db9-8f48-c348d6e47475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.159444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04befd12-ca47-4db9-8f48-c348d6e47475 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.159444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04befd12-ca47-4db9-8f48-c348d6e47475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.159444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04befd12-ca47-4db9-8f48-c348d6e47475 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.159444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04befd12-ca47-4db9-8f48-c348d6e47475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.159444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04befd12-ca47-4db9-8f48-c348d6e47475 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:23.172977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bc810-d3e3-4855-951b-5beaa68a27b6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.195334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1fb62-1a1b-4de2-a92d-3a3fe6afa61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.196364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1fb62-1a1b-4de2-a92d-3a3fe6afa61f admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.199195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1fb62-1a1b-4de2-a92d-3a3fe6afa61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.200013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1fb62-1a1b-4de2-a92d-3a3fe6afa61f admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.200671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1fb62-1a1b-4de2-a92d-3a3fe6afa61f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.205799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1fb62-1a1b-4de2-a92d-3a3fe6afa61f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:23.295083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1264/2520] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:48:23 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:23.300433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04befd12-ca47-4db9-8f48-c348d6e47475 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.324342 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1257/2521] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:48:23 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:23.329669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1fb62-1a1b-4de2-a92d-3a3fe6afa61f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.388975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04befd12-ca47-4db9-8f48-c348d6e47475 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.481885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c791881-173f-4399-be88-8a5b1795d1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.487150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c791881-173f-4399-be88-8a5b1795d1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.487519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c791881-173f-4399-be88-8a5b1795d1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.488125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c791881-173f-4399-be88-8a5b1795d1f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.488844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c791881-173f-4399-be88-8a5b1795d1f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.491631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c791881-173f-4399-be88-8a5b1795d1f3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:23.503593 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:23.535796 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1fb62-1a1b-4de2-a92d-3a3fe6afa61f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.543562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5c24a-eb9a-45bb-bd40-6952f6228ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.544224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5c24a-eb9a-45bb-bd40-6952f6228ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.544513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5c24a-eb9a-45bb-bd40-6952f6228ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.546421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5c24a-eb9a-45bb-bd40-6952f6228ae3 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.546421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a5c24a-eb9a-45bb-bd40-6952f6228ae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.548232 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a5c24a-eb9a-45bb-bd40-6952f6228ae3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:23.597195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:23.604129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1265/2522] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:48:23 2022] POST /identity/v3/users => generated 287 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:23.608461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c791881-173f-4399-be88-8a5b1795d1f3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.632574 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c791881-173f-4399-be88-8a5b1795d1f3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.639819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb30e0-b02f-43b0-bda7-0caac417d50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.640508 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb30e0-b02f-43b0-bda7-0caac417d50e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.640798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb30e0-b02f-43b0-bda7-0caac417d50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.643034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb30e0-b02f-43b0-bda7-0caac417d50e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.643034 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb30e0-b02f-43b0-bda7-0caac417d50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.653087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb30e0-b02f-43b0-bda7-0caac417d50e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:23.692950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1266/2523] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:48:23.697206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb30e0-b02f-43b0-bda7-0caac417d50e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.704405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1258/2524] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:48:23 2022] POST /identity/v3/users => generated 287 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:23.705580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5c24a-eb9a-45bb-bd40-6952f6228ae3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.717970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5c24a-eb9a-45bb-bd40-6952f6228ae3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.723062 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841af0c8-eab2-4054-96b0-0587eec7a1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.727707 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841af0c8-eab2-4054-96b0-0587eec7a1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.728058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841af0c8-eab2-4054-96b0-0587eec7a1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.728345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841af0c8-eab2-4054-96b0-0587eec7a1f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.729689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841af0c8-eab2-4054-96b0-0587eec7a1f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.731016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841af0c8-eab2-4054-96b0-0587eec7a1f6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:23.731340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb30e0-b02f-43b0-bda7-0caac417d50e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.746606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1259/2525] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:23.749854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70dc6fd-a75c-4438-a346-51c53041d5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.750607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc6fd-a75c-4438-a346-51c53041d5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.750894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc6fd-a75c-4438-a346-51c53041d5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.751265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc6fd-a75c-4438-a346-51c53041d5a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.752276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70dc6fd-a75c-4438-a346-51c53041d5a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.753708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70dc6fd-a75c-4438-a346-51c53041d5a5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:23.756171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841af0c8-eab2-4054-96b0-0587eec7a1f6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.811974 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1267/2526] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:48:23.815517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70dc6fd-a75c-4438-a346-51c53041d5a5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.822588 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841af0c8-eab2-4054-96b0-0587eec7a1f6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.832846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f70b-dcf4-4796-9942-e6da62ab0f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.833886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f70b-dcf4-4796-9942-e6da62ab0f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.836446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f70b-dcf4-4796-9942-e6da62ab0f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.837470 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f70b-dcf4-4796-9942-e6da62ab0f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.838439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3f70b-dcf4-4796-9942-e6da62ab0f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.840215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70dc6fd-a75c-4438-a346-51c53041d5a5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.856574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3f70b-dcf4-4796-9942-e6da62ab0f75 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:23.871583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750769d-4324-4b63-a04c-0d07fb5d428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.872897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750769d-4324-4b63-a04c-0d07fb5d428e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.886439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750769d-4324-4b63-a04c-0d07fb5d428e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.886831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750769d-4324-4b63-a04c-0d07fb5d428e admin admin] PATH_INFO: `/v3/projects/23f6b9048c7f48e0bdc37ee1d19df177/users/51819962789845c8b56ed869eb28c54e/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.900345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1260/2527] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:23.901835 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f70b-dcf4-4796-9942-e6da62ab0f75 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:23.915139 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f70b-dcf4-4796-9942-e6da62ab0f75 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:23.929852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bad918-e290-489f-bd25-54f8efa0db0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:23.931831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bad918-e290-489f-bd25-54f8efa0db0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:23.932379 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bad918-e290-489f-bd25-54f8efa0db0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:23.932893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bad918-e290-489f-bd25-54f8efa0db0b admin admin] PATH_INFO: `/v3/projects/f9658393adb548a681ee2fa3853b8410/users/dfebf6967afb405bbcf28335be89e521/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:23.938079 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750769d-4324-4b63-a04c-0d07fb5d428e admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f6b9048c7f48e0bdc37ee1d19df177, user_id=51819962789845c8b56ed869eb28c54e, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.941118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750769d-4324-4b63-a04c-0d07fb5d428e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:23.967630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bad918-e290-489f-bd25-54f8efa0db0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9658393adb548a681ee2fa3853b8410, user_id=dfebf6967afb405bbcf28335be89e521, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:23.973032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bad918-e290-489f-bd25-54f8efa0db0b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:24.023941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1268/2528] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:23 2022] PUT /identity/v3/projects/23f6b9048c7f48e0bdc37ee1d19df177/users/51819962789845c8b56ed869eb28c54e/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:24.034208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccaa16-d511-4c64-b65a-e64d859427d5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:24.034555 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccaa16-d511-4c64-b65a-e64d859427d5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:24.034867 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccaa16-d511-4c64-b65a-e64d859427d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:24.056258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1261/2529] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:23 2022] PUT /identity/v3/projects/f9658393adb548a681ee2fa3853b8410/users/dfebf6967afb405bbcf28335be89e521/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:24.061755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401f9bb-56c1-4d3f-9d6e-4d3e4edbeee9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:24.061935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401f9bb-56c1-4d3f-9d6e-4d3e4edbeee9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:24.062222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401f9bb-56c1-4d3f-9d6e-4d3e4edbeee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:24.087269 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-10ccaa16-d511-4c64-b65a-e64d859427d5 None None] MFA Rules not processed for user `51819962789845c8b56ed869eb28c54e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:24.088864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-10ccaa16-d511-4c64-b65a-e64d859427d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:24.124322 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7401f9bb-56c1-4d3f-9d6e-4d3e4edbeee9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:24.125449 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7401f9bb-56c1-4d3f-9d6e-4d3e4edbeee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:24.237086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1269/2530] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:24.238377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccaa16-d511-4c64-b65a-e64d859427d5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:24.342304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1262/2531] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:24.352495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401f9bb-56c1-4d3f-9d6e-4d3e4edbeee9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:24.354126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7401f9bb-56c1-4d3f-9d6e-4d3e4edbeee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:24.441823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccaa16-d511-4c64-b65a-e64d859427d5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:24.448753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd88524-c683-4827-8db1-79070aa01111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:24.449836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd88524-c683-4827-8db1-79070aa01111 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:24.450154 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd88524-c683-4827-8db1-79070aa01111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:24.450759 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd88524-c683-4827-8db1-79070aa01111 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:24.452118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd88524-c683-4827-8db1-79070aa01111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:24.455117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd88524-c683-4827-8db1-79070aa01111 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:24.519315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401f9bb-56c1-4d3f-9d6e-4d3e4edbeee9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:24.524585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e665928-b475-4798-9a98-bbb6fba65133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:24.525000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e665928-b475-4798-9a98-bbb6fba65133 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:24.525336 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e665928-b475-4798-9a98-bbb6fba65133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:24.525692 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e665928-b475-4798-9a98-bbb6fba65133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:24.539594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e665928-b475-4798-9a98-bbb6fba65133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:24.541783 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e665928-b475-4798-9a98-bbb6fba65133 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:24.562917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1263/2532] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:48:24.659955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e665928-b475-4798-9a98-bbb6fba65133 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:24.670157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1270/2533] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:48:24 2022] POST /identity/v3/projects => generated 371 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:24.673073 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e665928-b475-4798-9a98-bbb6fba65133 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:24.760887 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:24.892921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606e889-984d-47fb-b639-9f239db72aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:24.892921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e889-984d-47fb-b639-9f239db72aae admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:24.892921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e889-984d-47fb-b639-9f239db72aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:24.892921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e889-984d-47fb-b639-9f239db72aae admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:24.893310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606e889-984d-47fb-b639-9f239db72aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:24.893310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606e889-984d-47fb-b639-9f239db72aae admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:25.005685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1264/2534] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:48:24 2022] POST /identity/v3/users => generated 292 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:25.010111 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd88524-c683-4827-8db1-79070aa01111 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:25.029737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd88524-c683-4827-8db1-79070aa01111 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:25.050482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98434ea9-4a05-425d-9bda-9ab5508e6e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:25.051130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98434ea9-4a05-425d-9bda-9ab5508e6e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:25.051416 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98434ea9-4a05-425d-9bda-9ab5508e6e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:25.051703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98434ea9-4a05-425d-9bda-9ab5508e6e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:25.052431 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98434ea9-4a05-425d-9bda-9ab5508e6e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:25.054713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98434ea9-4a05-425d-9bda-9ab5508e6e4f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:25.097627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1271/2535] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:25.105217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606e889-984d-47fb-b639-9f239db72aae admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:25.127876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606e889-984d-47fb-b639-9f239db72aae admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:25.139608 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306178a-5c55-4ccc-b27d-5b2b9f60562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:25.141531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306178a-5c55-4ccc-b27d-5b2b9f60562d admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:25.141531 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306178a-5c55-4ccc-b27d-5b2b9f60562d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:25.142186 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306178a-5c55-4ccc-b27d-5b2b9f60562d admin admin] PATH_INFO: `/v3/projects/748d705bbef3445fbca130698621d7b7/users/ca12953e6a8041149ec3b27bf18d1491/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:25.208998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306178a-5c55-4ccc-b27d-5b2b9f60562d admin admin] RBAC: Authorizing `identity:create_grant(project_id=748d705bbef3445fbca130698621d7b7, user_id=ca12953e6a8041149ec3b27bf18d1491, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:25.226558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306178a-5c55-4ccc-b27d-5b2b9f60562d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:25.845525 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1265/2536] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:25 2022] PUT /identity/v3/projects/748d705bbef3445fbca130698621d7b7/users/ca12953e6a8041149ec3b27bf18d1491/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:48:25.849297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98434ea9-4a05-425d-9bda-9ab5508e6e4f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:25.867130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98434ea9-4a05-425d-9bda-9ab5508e6e4f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:25.903153 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:26.091014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12941dc0-0bb2-4e6b-850a-3462aeba9473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:26.092799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12941dc0-0bb2-4e6b-850a-3462aeba9473 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:26.093340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12941dc0-0bb2-4e6b-850a-3462aeba9473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:26.093832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12941dc0-0bb2-4e6b-850a-3462aeba9473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:26.094819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12941dc0-0bb2-4e6b-850a-3462aeba9473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:26.097133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12941dc0-0bb2-4e6b-850a-3462aeba9473 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:26.117917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1272/2537] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:26.122191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306178a-5c55-4ccc-b27d-5b2b9f60562d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:26.137811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306178a-5c55-4ccc-b27d-5b2b9f60562d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:26.160439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c6f1f-3996-48fc-91a8-e729286ee599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:26.161179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c6f1f-3996-48fc-91a8-e729286ee599 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:26.161659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c6f1f-3996-48fc-91a8-e729286ee599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:26.162125 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c6f1f-3996-48fc-91a8-e729286ee599 admin admin] PATH_INFO: `/v3/projects/748d705bbef3445fbca130698621d7b7/users/ca12953e6a8041149ec3b27bf18d1491/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:26.173260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c6f1f-3996-48fc-91a8-e729286ee599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=748d705bbef3445fbca130698621d7b7, user_id=ca12953e6a8041149ec3b27bf18d1491, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:26.184226 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c6f1f-3996-48fc-91a8-e729286ee599 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:26.204451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1266/2538] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:26 2022] PUT /identity/v3/projects/748d705bbef3445fbca130698621d7b7/users/ca12953e6a8041149ec3b27bf18d1491/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:26.216247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404345d0-2fd7-4dc4-a4fc-f7e9071de6a9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:26.217354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404345d0-2fd7-4dc4-a4fc-f7e9071de6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:26.217354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404345d0-2fd7-4dc4-a4fc-f7e9071de6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:26.282727 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-404345d0-2fd7-4dc4-a4fc-f7e9071de6a9 None None] MFA Rules not processed for user `ca12953e6a8041149ec3b27bf18d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:26.284132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-404345d0-2fd7-4dc4-a4fc-f7e9071de6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:26.457291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1273/2539] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:26.478937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c6f1f-3996-48fc-91a8-e729286ee599 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:26.478937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-210c6f1f-3996-48fc-91a8-e729286ee599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:26.563853 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c6f1f-3996-48fc-91a8-e729286ee599 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:26.569505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7bb44-c74c-48c2-bd0c-f178e5a1aa99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:26.570193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7bb44-c74c-48c2-bd0c-f178e5a1aa99 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:26.570682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7bb44-c74c-48c2-bd0c-f178e5a1aa99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:26.570961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7bb44-c74c-48c2-bd0c-f178e5a1aa99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:26.583392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7bb44-c74c-48c2-bd0c-f178e5a1aa99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:26.585646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7bb44-c74c-48c2-bd0c-f178e5a1aa99 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:26.596509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1267/2540] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:27.137642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404345d0-2fd7-4dc4-a4fc-f7e9071de6a9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:27.151351 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404345d0-2fd7-4dc4-a4fc-f7e9071de6a9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:27.158187 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f45119-59d8-4f36-b362-6318f7c877ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:27.159325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f45119-59d8-4f36-b362-6318f7c877ea service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:27.159761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f45119-59d8-4f36-b362-6318f7c877ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:27.160393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f45119-59d8-4f36-b362-6318f7c877ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:27.250929 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f45119-59d8-4f36-b362-6318f7c877ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:27.256468 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f45119-59d8-4f36-b362-6318f7c877ea service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:27.302427 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1274/2541] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:27.421407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf9906-0590-4445-8797-c33e79b3b475 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:27.422891 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf9906-0590-4445-8797-c33e79b3b475 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:27.423575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf9906-0590-4445-8797-c33e79b3b475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:27.490012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6bdf9906-0590-4445-8797-c33e79b3b475 None None] MFA Rules not processed for user `dfebf6967afb405bbcf28335be89e521`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:27.501554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6bdf9906-0590-4445-8797-c33e79b3b475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:27.736935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1268/2542] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:27.738807 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f45119-59d8-4f36-b362-6318f7c877ea service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:27.849748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:27.893305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f45119-59d8-4f36-b362-6318f7c877ea service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:27.903539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb20b-c76e-43ac-a102-e843dbf97d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:27.907172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb20b-c76e-43ac-a102-e843dbf97d80 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:27.907172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb20b-c76e-43ac-a102-e843dbf97d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:27.907172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb20b-c76e-43ac-a102-e843dbf97d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:27.907172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacb20b-c76e-43ac-a102-e843dbf97d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:27.921907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacb20b-c76e-43ac-a102-e843dbf97d80 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:28.061475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1275/2543] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:48:27 2022] POST /identity/v3/projects => generated 371 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:28.071592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf9906-0590-4445-8797-c33e79b3b475 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:28.087131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf9906-0590-4445-8797-c33e79b3b475 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:28.096982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:28.161634 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da458c6-03ef-44d6-88b0-8cb504248e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:28.162446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da458c6-03ef-44d6-88b0-8cb504248e1e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:28.163444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da458c6-03ef-44d6-88b0-8cb504248e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:28.163921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da458c6-03ef-44d6-88b0-8cb504248e1e admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:28.165035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da458c6-03ef-44d6-88b0-8cb504248e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:28.167391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da458c6-03ef-44d6-88b0-8cb504248e1e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:28.316724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1269/2544] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:48:28 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:28.320910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb20b-c76e-43ac-a102-e843dbf97d80 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:28.335670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb20b-c76e-43ac-a102-e843dbf97d80 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:28.342054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8eff8-5c98-4242-b637-bdf1ab6d106b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:28.342986 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8eff8-5c98-4242-b637-bdf1ab6d106b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:28.343567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8eff8-5c98-4242-b637-bdf1ab6d106b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:28.344151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8eff8-5c98-4242-b637-bdf1ab6d106b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:28.345183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8eff8-5c98-4242-b637-bdf1ab6d106b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:28.347815 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8eff8-5c98-4242-b637-bdf1ab6d106b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:28.362945 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1276/2545] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:28.367550 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da458c6-03ef-44d6-88b0-8cb504248e1e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:28.385246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da458c6-03ef-44d6-88b0-8cb504248e1e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:28.391705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bb7ed-b392-48d0-981b-c6f8acb7893b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:28.392491 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bb7ed-b392-48d0-981b-c6f8acb7893b admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:28.392704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bb7ed-b392-48d0-981b-c6f8acb7893b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:28.393171 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bb7ed-b392-48d0-981b-c6f8acb7893b admin admin] PATH_INFO: `/v3/projects/836eb0054147498ab1559b0bc8384008/users/46ec15ca0bf44abfa09f4e8d4a282eb9/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:28.415516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bb7ed-b392-48d0-981b-c6f8acb7893b admin admin] RBAC: Authorizing `identity:create_grant(project_id=836eb0054147498ab1559b0bc8384008, user_id=46ec15ca0bf44abfa09f4e8d4a282eb9, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:28.418404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bb7ed-b392-48d0-981b-c6f8acb7893b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:28.514548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1270/2546] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:28 2022] PUT /identity/v3/projects/836eb0054147498ab1559b0bc8384008/users/46ec15ca0bf44abfa09f4e8d4a282eb9/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:28.518317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8eff8-5c98-4242-b637-bdf1ab6d106b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:28.531237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8eff8-5c98-4242-b637-bdf1ab6d106b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:28.613082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72300fe8-32fa-4843-be8e-6caf3f9ebd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:28.613707 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72300fe8-32fa-4843-be8e-6caf3f9ebd82 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:28.613991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72300fe8-32fa-4843-be8e-6caf3f9ebd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:28.614361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72300fe8-32fa-4843-be8e-6caf3f9ebd82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:28.615073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72300fe8-32fa-4843-be8e-6caf3f9ebd82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:28.616795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72300fe8-32fa-4843-be8e-6caf3f9ebd82 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:28.633827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1277/2547] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:28.638034 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bb7ed-b392-48d0-981b-c6f8acb7893b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:28.651359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bb7ed-b392-48d0-981b-c6f8acb7893b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:28.661425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c5b8a-9895-437d-9afa-14c13a8d6f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:28.663256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c5b8a-9895-437d-9afa-14c13a8d6f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:28.664098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c5b8a-9895-437d-9afa-14c13a8d6f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:28.664527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c5b8a-9895-437d-9afa-14c13a8d6f83 admin admin] PATH_INFO: `/v3/projects/836eb0054147498ab1559b0bc8384008/users/46ec15ca0bf44abfa09f4e8d4a282eb9/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:28.667798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c5b8a-9895-437d-9afa-14c13a8d6f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=836eb0054147498ab1559b0bc8384008, user_id=46ec15ca0bf44abfa09f4e8d4a282eb9, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:28.672058 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c5b8a-9895-437d-9afa-14c13a8d6f83 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:28.698195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1271/2548] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:28 2022] PUT /identity/v3/projects/836eb0054147498ab1559b0bc8384008/users/46ec15ca0bf44abfa09f4e8d4a282eb9/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:31.413573 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0841d-92be-445f-870b-06540cc99d3f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:31.414329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0841d-92be-445f-870b-06540cc99d3f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:31.414329 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0841d-92be-445f-870b-06540cc99d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:31.463684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-55f0841d-92be-445f-870b-06540cc99d3f None None] MFA Rules not processed for user `46ec15ca0bf44abfa09f4e8d4a282eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:31.464482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-55f0841d-92be-445f-870b-06540cc99d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:31.564475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1278/2549] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:48:31.579766 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c5b8a-9895-437d-9afa-14c13a8d6f83 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:31.658813 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c5b8a-9895-437d-9afa-14c13a8d6f83 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:31.662756 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edee37-7371-41f7-b6f1-f1b013c06d6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:31.663590 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edee37-7371-41f7-b6f1-f1b013c06d6a service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:31.663869 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edee37-7371-41f7-b6f1-f1b013c06d6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:31.664188 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edee37-7371-41f7-b6f1-f1b013c06d6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:31.742979 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edee37-7371-41f7-b6f1-f1b013c06d6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:31.744735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edee37-7371-41f7-b6f1-f1b013c06d6a service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:31.754468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1272/2550] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:35.385536 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc556bf-9110-4382-840e-be53ca36f229 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:35.386222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc556bf-9110-4382-840e-be53ca36f229 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:35.386222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc556bf-9110-4382-840e-be53ca36f229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:35.407202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2cc556bf-9110-4382-840e-be53ca36f229 None None] MFA Rules not processed for user `2979498fe4614879800a313e1a932fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:35.408446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2cc556bf-9110-4382-840e-be53ca36f229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:35.438166 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:35.464713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1279/2551] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:35.713104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edee37-7371-41f7-b6f1-f1b013c06d6a service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:35.721836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edee37-7371-41f7-b6f1-f1b013c06d6a service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:35.726367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f567f2-0aab-4cae-abf9-0c836267fa39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:35.726921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f567f2-0aab-4cae-abf9-0c836267fa39 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:35.727205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f567f2-0aab-4cae-abf9-0c836267fa39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:35.727479 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f567f2-0aab-4cae-abf9-0c836267fa39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:35.729381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc556bf-9110-4382-840e-be53ca36f229 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:35.730924 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2cc556bf-9110-4382-840e-be53ca36f229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:35.737233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f567f2-0aab-4cae-abf9-0c836267fa39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:35.738655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f567f2-0aab-4cae-abf9-0c836267fa39 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:35.748791 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1273/2552] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:35.788891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc556bf-9110-4382-840e-be53ca36f229 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:35.792448 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abb84d-3e51-4686-bfee-ad9c20a703b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:35.793061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abb84d-3e51-4686-bfee-ad9c20a703b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:35.793342 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abb84d-3e51-4686-bfee-ad9c20a703b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:35.793615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abb84d-3e51-4686-bfee-ad9c20a703b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:35.795350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-70abb84d-3e51-4686-bfee-ad9c20a703b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:35.805128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abb84d-3e51-4686-bfee-ad9c20a703b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:35.806552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abb84d-3e51-4686-bfee-ad9c20a703b7 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:35.814501 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1280/2553] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:48:38.825255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa93a26-f58e-43a9-bb1e-60d479e8cd82 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:38.825741 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa93a26-f58e-43a9-bb1e-60d479e8cd82 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:38.825848 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa93a26-f58e-43a9-bb1e-60d479e8cd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:38.841398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:38.848148 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3fa93a26-f58e-43a9-bb1e-60d479e8cd82 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:38.849529 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3fa93a26-f58e-43a9-bb1e-60d479e8cd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:38.908849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1274/2554] 10.222.0.64 () {58 vars in 967 bytes} [Thu Aug 4 07:48:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:38.924281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abb84d-3e51-4686-bfee-ad9c20a703b7 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:38.926631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-70abb84d-3e51-4686-bfee-ad9c20a703b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:39.017851 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abb84d-3e51-4686-bfee-ad9c20a703b7 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:39.023097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbca7c-b746-4d06-96db-fb25cece105c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:39.023717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbca7c-b746-4d06-96db-fb25cece105c service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:39.024021 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbca7c-b746-4d06-96db-fb25cece105c service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:39.024295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbca7c-b746-4d06-96db-fb25cece105c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:39.032772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbca7c-b746-4d06-96db-fb25cece105c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:39.034530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbca7c-b746-4d06-96db-fb25cece105c service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:39.044839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1281/2555] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:48:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:48:39.663214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c881b-71f5-4e45-92de-ff833bd3fc49 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:39.663214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c881b-71f5-4e45-92de-ff833bd3fc49 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:39.663214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c881b-71f5-4e45-92de-ff833bd3fc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:39.680390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ad5c881b-71f5-4e45-92de-ff833bd3fc49 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:39.681594 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ad5c881b-71f5-4e45-92de-ff833bd3fc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:39.688841 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1275/2556] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:48:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:39.702266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbca7c-b746-4d06-96db-fb25cece105c service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:39.713680 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbca7c-b746-4d06-96db-fb25cece105c service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:39.722441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea4f77-c611-4b2a-a15d-e25d68272958 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:39.723549 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea4f77-c611-4b2a-a15d-e25d68272958 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:39.724104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea4f77-c611-4b2a-a15d-e25d68272958 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:39.724306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea4f77-c611-4b2a-a15d-e25d68272958 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:39.735691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea4f77-c611-4b2a-a15d-e25d68272958 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:39.737844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea4f77-c611-4b2a-a15d-e25d68272958 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:39.749011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1282/2557] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:48:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:48:49.674527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=25 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:49.745241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=74 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:53.172744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c881b-71f5-4e45-92de-ff833bd3fc49 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:53.242233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c881b-71f5-4e45-92de-ff833bd3fc49 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:53.247433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:53.248119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:53.248406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:53.248688 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] PATH_INFO: `/v3/users/dfebf6967afb405bbcf28335be89e521` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:53.249934 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfebf6967afb405bbcf28335be89e521)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:53.251906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:53.438255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfebf6967afb405bbcf28335be89e521'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:53.439089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfebf6967afb405bbcf28335be89e521'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:53.439799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfebf6967afb405bbcf28335be89e521'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:53.455079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfebf6967afb405bbcf28335be89e521'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:53.501989 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfebf6967afb405bbcf28335be89e521'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:53.514905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1276/2558] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:48:53 2022] DELETE /identity/v3/users/dfebf6967afb405bbcf28335be89e521 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:53.778790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea4f77-c611-4b2a-a15d-e25d68272958 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:53.791526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea4f77-c611-4b2a-a15d-e25d68272958 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:53.853314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:53.854194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:53.854542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:53.854882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] PATH_INFO: `/v3/projects/f9658393adb548a681ee2fa3853b8410` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:53.856260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9658393adb548a681ee2fa3853b8410)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:53.858317 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:53.979464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9658393adb548a681ee2fa3853b8410', 'request_id': 'req-5414ae90-ea70-4375-9e22-775699912e2c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:54.056217 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9658393adb548a681ee2fa3853b8410', 'request_id': 'req-5414ae90-ea70-4375-9e22-775699912e2c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:54.062392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9658393adb548a681ee2fa3853b8410', 'request_id': 'req-5414ae90-ea70-4375-9e22-775699912e2c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:54.073752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] The token cache is being invalidate because project f9658393adb548a681ee2fa3853b8410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:48:54.074038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:54.074668 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:54.076297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1283/2559] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:48:53 2022] DELETE /identity/v3/projects/f9658393adb548a681ee2fa3853b8410 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:54.080677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:54.082078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:54.095189 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef0fad-8a9b-423d-bf29-e8c06e647a95 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:54.151303 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308eba80-5da6-4324-b186-32016a22ee52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:54.151948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eba80-5da6-4324-b186-32016a22ee52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:54.152265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eba80-5da6-4324-b186-32016a22ee52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:54.152543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eba80-5da6-4324-b186-32016a22ee52 admin admin] PATH_INFO: `/v3/users/46ec15ca0bf44abfa09f4e8d4a282eb9` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:54.153716 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308eba80-5da6-4324-b186-32016a22ee52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46ec15ca0bf44abfa09f4e8d4a282eb9)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:54.155630 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308eba80-5da6-4324-b186-32016a22ee52 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:54.178736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:54.339582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-308eba80-5da6-4324-b186-32016a22ee52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46ec15ca0bf44abfa09f4e8d4a282eb9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:54.340292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-308eba80-5da6-4324-b186-32016a22ee52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46ec15ca0bf44abfa09f4e8d4a282eb9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:54.340987 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-308eba80-5da6-4324-b186-32016a22ee52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46ec15ca0bf44abfa09f4e8d4a282eb9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:54.353535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-308eba80-5da6-4324-b186-32016a22ee52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46ec15ca0bf44abfa09f4e8d4a282eb9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:54.456378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-308eba80-5da6-4324-b186-32016a22ee52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46ec15ca0bf44abfa09f4e8d4a282eb9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:54.469861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1277/2560] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:48:54 2022] DELETE /identity/v3/users/46ec15ca0bf44abfa09f4e8d4a282eb9 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:54.744356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:54.745814 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:54.760328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414ae90-ea70-4375-9e22-775699912e2c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:54.817918 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:54.818617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:54.818926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:54.819201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] PATH_INFO: `/v3/projects/836eb0054147498ab1559b0bc8384008` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:54.820365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=836eb0054147498ab1559b0bc8384008)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:54.822310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:54.952285 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:54.965036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '836eb0054147498ab1559b0bc8384008', 'request_id': 'req-3b640de1-d99a-485d-bd32-678fff3f4270'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:55.060781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '836eb0054147498ab1559b0bc8384008', 'request_id': 'req-3b640de1-d99a-485d-bd32-678fff3f4270'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:55.068856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '836eb0054147498ab1559b0bc8384008', 'request_id': 'req-3b640de1-d99a-485d-bd32-678fff3f4270'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:55.080547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] The token cache is being invalidate because project 836eb0054147498ab1559b0bc8384008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:48:55.080837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:55.081490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b640de1-d99a-485d-bd32-678fff3f4270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:55.083335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1284/2561] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:48:54 2022] DELETE /identity/v3/projects/836eb0054147498ab1559b0bc8384008 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:55.088110 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e89d-bb01-47cb-a9a4-d9f2a897ddb5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:55.088569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e89d-bb01-47cb-a9a4-d9f2a897ddb5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:55.088929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e89d-bb01-47cb-a9a4-d9f2a897ddb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:55.122614 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ddb5e89d-bb01-47cb-a9a4-d9f2a897ddb5 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:55.123961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ddb5e89d-bb01-47cb-a9a4-d9f2a897ddb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:55.194653 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1278/2562] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:55.203247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7feb56-e044-4316-ae2e-e92d1308a098 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:55.203532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7feb56-e044-4316-ae2e-e92d1308a098 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:55.204101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7feb56-e044-4316-ae2e-e92d1308a098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:55.225366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5d7feb56-e044-4316-ae2e-e92d1308a098 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:55.226547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5d7feb56-e044-4316-ae2e-e92d1308a098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:55.234323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1285/2563] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:55.238533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5e89d-bb01-47cb-a9a4-d9f2a897ddb5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:55.248084 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5e89d-bb01-47cb-a9a4-d9f2a897ddb5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:55.252600 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905d9ff-01cd-4122-a6be-556840ddc966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:55.253206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905d9ff-01cd-4122-a6be-556840ddc966 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:55.253487 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905d9ff-01cd-4122-a6be-556840ddc966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:55.253763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905d9ff-01cd-4122-a6be-556840ddc966 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:55.254449 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905d9ff-01cd-4122-a6be-556840ddc966 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:48:55.254842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905d9ff-01cd-4122-a6be-556840ddc966 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:55.256523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905d9ff-01cd-4122-a6be-556840ddc966 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:55.268786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1279/2564] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:48:55.273916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7feb56-e044-4316-ae2e-e92d1308a098 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:55.284375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7feb56-e044-4316-ae2e-e92d1308a098 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:55.289702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d7475-a96b-4c6b-826f-87c5181fab61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:55.293189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d7475-a96b-4c6b-826f-87c5181fab61 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:55.293189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d7475-a96b-4c6b-826f-87c5181fab61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:55.293189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d7475-a96b-4c6b-826f-87c5181fab61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:55.293189 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d7475-a96b-4c6b-826f-87c5181fab61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:55.293588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d7475-a96b-4c6b-826f-87c5181fab61 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:55.381942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1286/2565] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:48:55 2022] POST /identity/v3/projects => generated 359 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:48:55.386152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905d9ff-01cd-4122-a6be-556840ddc966 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:55.397607 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905d9ff-01cd-4122-a6be-556840ddc966 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:55.453056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf828cb-d113-49bc-94ac-86834952615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:55.453920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf828cb-d113-49bc-94ac-86834952615a admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:55.454220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf828cb-d113-49bc-94ac-86834952615a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:55.454499 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf828cb-d113-49bc-94ac-86834952615a admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:55.455181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf828cb-d113-49bc-94ac-86834952615a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:55.457142 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf828cb-d113-49bc-94ac-86834952615a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:55.552174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1280/2566] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:48:55 2022] POST /identity/v3/users => generated 280 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:55.556519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d7475-a96b-4c6b-826f-87c5181fab61 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:55.567281 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d7475-a96b-4c6b-826f-87c5181fab61 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:55.573069 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb159296-9423-482d-a435-86acf421db31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:55.573674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb159296-9423-482d-a435-86acf421db31 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:55.573955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb159296-9423-482d-a435-86acf421db31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:55.574230 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb159296-9423-482d-a435-86acf421db31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:55.574965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb159296-9423-482d-a435-86acf421db31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:55.576784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb159296-9423-482d-a435-86acf421db31 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:55.588308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1287/2567] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:55.592613 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf828cb-d113-49bc-94ac-86834952615a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:55.602718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf828cb-d113-49bc-94ac-86834952615a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:55.607241 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfa646-efad-4bc4-b32d-f104d28f442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:55.607842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa646-efad-4bc4-b32d-f104d28f442d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:55.608162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa646-efad-4bc4-b32d-f104d28f442d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:55.608438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa646-efad-4bc4-b32d-f104d28f442d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:55.609141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfa646-efad-4bc4-b32d-f104d28f442d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:55.610720 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfa646-efad-4bc4-b32d-f104d28f442d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:55.621313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1281/2568] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:55.627119 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb159296-9423-482d-a435-86acf421db31 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:55.637532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb159296-9423-482d-a435-86acf421db31 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:55.642602 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb67692-1a83-4058-aef4-3e17431ef4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:55.643256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb67692-1a83-4058-aef4-3e17431ef4f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:55.643538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb67692-1a83-4058-aef4-3e17431ef4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:55.643816 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb67692-1a83-4058-aef4-3e17431ef4f2 admin admin] PATH_INFO: `/v3/projects/bc0146a71018428293ecde525b2a5baf/users/10be64dd95d143b2817daddab33a147f/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:55.658865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb67692-1a83-4058-aef4-3e17431ef4f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc0146a71018428293ecde525b2a5baf, user_id=10be64dd95d143b2817daddab33a147f, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:55.661644 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb67692-1a83-4058-aef4-3e17431ef4f2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:55.752587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1288/2569] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:55 2022] PUT /identity/v3/projects/bc0146a71018428293ecde525b2a5baf/users/10be64dd95d143b2817daddab33a147f/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:55.763524 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f5b00-3cea-4ab5-9bf3-a7f54ee379cd None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:55.763836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f5b00-3cea-4ab5-9bf3-a7f54ee379cd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:55.764143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f5b00-3cea-4ab5-9bf3-a7f54ee379cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:55.772934 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:55.791756 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-833f5b00-3cea-4ab5-9bf3-a7f54ee379cd None None] MFA Rules not processed for user `10be64dd95d143b2817daddab33a147f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:55.793088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-833f5b00-3cea-4ab5-9bf3-a7f54ee379cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:55.859975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1282/2570] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:55.897915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb67692-1a83-4058-aef4-3e17431ef4f2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:55.899668 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4cb67692-1a83-4058-aef4-3e17431ef4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:55.956621 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb67692-1a83-4058-aef4-3e17431ef4f2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:55.960310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10ef18-ce79-454c-aeb0-dd13e3b122be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:55.960983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10ef18-ce79-454c-aeb0-dd13e3b122be service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:55.961266 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10ef18-ce79-454c-aeb0-dd13e3b122be service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:55.961531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10ef18-ce79-454c-aeb0-dd13e3b122be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:55.969888 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10ef18-ce79-454c-aeb0-dd13e3b122be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:55.971311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10ef18-ce79-454c-aeb0-dd13e3b122be service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:55.980157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1289/2571] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:55.996006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f5b00-3cea-4ab5-9bf3-a7f54ee379cd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:56.057826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f5b00-3cea-4ab5-9bf3-a7f54ee379cd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:56.062331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:56.063015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:56.063296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:56.063570 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] PATH_INFO: `/v3/users/10be64dd95d143b2817daddab33a147f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:56.064742 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] RBAC: Authorizing `identity:delete_user(user_id=10be64dd95d143b2817daddab33a147f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:56.066647 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:56.253211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10be64dd95d143b2817daddab33a147f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:56.253925 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10be64dd95d143b2817daddab33a147f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:56.254452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10be64dd95d143b2817daddab33a147f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:56.266509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10be64dd95d143b2817daddab33a147f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:56.307920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10be64dd95d143b2817daddab33a147f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:56.321311 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1283/2572] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:48:55 2022] DELETE /identity/v3/users/10be64dd95d143b2817daddab33a147f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:56.326371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc067fb-ab1f-4ca7-b406-7c83ff85c8f9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:56.326371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc067fb-ab1f-4ca7-b406-7c83ff85c8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:56.326754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc067fb-ab1f-4ca7-b406-7c83ff85c8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:56.347208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ddc067fb-ab1f-4ca7-b406-7c83ff85c8f9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:56.348432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ddc067fb-ab1f-4ca7-b406-7c83ff85c8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:56.400743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:56.409275 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1290/2573] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:56.420802 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:56.422270 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:56.439344 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:56.485154 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706de56-979b-4165-974c-1b19f3f2473c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:56.489120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66aadb-dd76-4dbe-81f3-d86ae62f624c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:56.492681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66aadb-dd76-4dbe-81f3-d86ae62f624c service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:56.492681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66aadb-dd76-4dbe-81f3-d86ae62f624c service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:56.492681 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66aadb-dd76-4dbe-81f3-d86ae62f624c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:56.500684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66aadb-dd76-4dbe-81f3-d86ae62f624c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:56.502175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66aadb-dd76-4dbe-81f3-d86ae62f624c service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:56.512289 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1284/2574] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:48:57.070133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc067fb-ab1f-4ca7-b406-7c83ff85c8f9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:57.071589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ddc067fb-ab1f-4ca7-b406-7c83ff85c8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:57.088103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc067fb-ab1f-4ca7-b406-7c83ff85c8f9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:57.094175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:57.098558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:57.098558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:57.098558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] PATH_INFO: `/v3/projects/bc0146a71018428293ecde525b2a5baf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:57.098558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc0146a71018428293ecde525b2a5baf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:57.101774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:57.222301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc0146a71018428293ecde525b2a5baf', 'request_id': 'req-b5c4b15d-85dd-467f-b4d9-d278c232eff4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:57.279224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc0146a71018428293ecde525b2a5baf', 'request_id': 'req-b5c4b15d-85dd-467f-b4d9-d278c232eff4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:57.285557 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc0146a71018428293ecde525b2a5baf', 'request_id': 'req-b5c4b15d-85dd-467f-b4d9-d278c232eff4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:57.296792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] The token cache is being invalidate because project bc0146a71018428293ecde525b2a5baf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:48:57.297103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:57.297797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5c4b15d-85dd-467f-b4d9-d278c232eff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:48:57.299751 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1291/2575] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:48:57 2022] DELETE /identity/v3/projects/bc0146a71018428293ecde525b2a5baf => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:48:57.304742 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e9ae-d84c-4315-8e8a-e4d0b4978237 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:57.305228 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e9ae-d84c-4315-8e8a-e4d0b4978237 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:57.305375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e9ae-d84c-4315-8e8a-e4d0b4978237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:57.326095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7ef6e9ae-d84c-4315-8e8a-e4d0b4978237 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:57.327297 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7ef6e9ae-d84c-4315-8e8a-e4d0b4978237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:57.387844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1285/2576] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:57.396697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73645584-532c-4cd6-b745-b515aec0e897 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:57.396978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73645584-532c-4cd6-b745-b515aec0e897 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:57.397250 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73645584-532c-4cd6-b745-b515aec0e897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:57.416766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-73645584-532c-4cd6-b745-b515aec0e897 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:57.417954 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-73645584-532c-4cd6-b745-b515aec0e897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:57.425856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1292/2577] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:57.430689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6e9ae-d84c-4315-8e8a-e4d0b4978237 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:57.440616 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6e9ae-d84c-4315-8e8a-e4d0b4978237 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:57.446038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfcf58-8dab-4037-8c3a-e602b40a288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:57.448343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfcf58-8dab-4037-8c3a-e602b40a288c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:57.448343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfcf58-8dab-4037-8c3a-e602b40a288c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:57.448343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfcf58-8dab-4037-8c3a-e602b40a288c admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:57.448343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfcf58-8dab-4037-8c3a-e602b40a288c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:48:57.448343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfcf58-8dab-4037-8c3a-e602b40a288c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:57.449862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfcf58-8dab-4037-8c3a-e602b40a288c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:57.461978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1286/2578] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:48:57.466314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73645584-532c-4cd6-b745-b515aec0e897 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:57.475535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73645584-532c-4cd6-b745-b515aec0e897 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:57.480197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fb8e9-94d6-47e7-a92a-b2060b830112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:57.480805 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fb8e9-94d6-47e7-a92a-b2060b830112 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:57.481088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fb8e9-94d6-47e7-a92a-b2060b830112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:57.481367 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fb8e9-94d6-47e7-a92a-b2060b830112 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:57.482037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138fb8e9-94d6-47e7-a92a-b2060b830112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:57.483942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138fb8e9-94d6-47e7-a92a-b2060b830112 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:57.606810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1293/2579] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:48:57 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:57.616730 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfcf58-8dab-4037-8c3a-e602b40a288c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:57.627040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfcf58-8dab-4037-8c3a-e602b40a288c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:57.688450 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acb4d6-566a-4f75-b262-cabfd68f318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:57.689161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acb4d6-566a-4f75-b262-cabfd68f318d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:57.689503 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acb4d6-566a-4f75-b262-cabfd68f318d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:57.689807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acb4d6-566a-4f75-b262-cabfd68f318d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:57.690486 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acb4d6-566a-4f75-b262-cabfd68f318d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:57.692454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acb4d6-566a-4f75-b262-cabfd68f318d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:57.821471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1287/2580] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:48:57 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:48:57.825596 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fb8e9-94d6-47e7-a92a-b2060b830112 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:57.836396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fb8e9-94d6-47e7-a92a-b2060b830112 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:57.840960 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f335b0e-70ad-4f10-bf44-07bede501058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:57.841565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335b0e-70ad-4f10-bf44-07bede501058 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:57.841846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335b0e-70ad-4f10-bf44-07bede501058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:57.842124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335b0e-70ad-4f10-bf44-07bede501058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:57.842833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f335b0e-70ad-4f10-bf44-07bede501058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:57.844460 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f335b0e-70ad-4f10-bf44-07bede501058 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:57.855032 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1294/2581] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:57.858832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acb4d6-566a-4f75-b262-cabfd68f318d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:57.870095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acb4d6-566a-4f75-b262-cabfd68f318d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:57.874894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeec075-8ace-49f6-aa1f-8e866f27d2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:57.875505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeec075-8ace-49f6-aa1f-8e866f27d2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:57.875795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeec075-8ace-49f6-aa1f-8e866f27d2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:57.876118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeec075-8ace-49f6-aa1f-8e866f27d2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:57.876815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeec075-8ace-49f6-aa1f-8e866f27d2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:57.878428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeec075-8ace-49f6-aa1f-8e866f27d2c3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:57.889602 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1288/2582] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:48:57.893710 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f335b0e-70ad-4f10-bf44-07bede501058 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:57.903767 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f335b0e-70ad-4f10-bf44-07bede501058 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:57.908277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e972a61-da01-495a-8e95-19d2996f22da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:57.908931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e972a61-da01-495a-8e95-19d2996f22da admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:57.909209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e972a61-da01-495a-8e95-19d2996f22da admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:57.909496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e972a61-da01-495a-8e95-19d2996f22da admin admin] PATH_INFO: `/v3/projects/b396c4245df1455bac98c4591a3f0772/users/88b91495fe4b4380972638e63523f195/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:57.924231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e972a61-da01-495a-8e95-19d2996f22da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b396c4245df1455bac98c4591a3f0772, user_id=88b91495fe4b4380972638e63523f195, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:57.926867 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e972a61-da01-495a-8e95-19d2996f22da admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:58.021030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1295/2583] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:48:57 2022] PUT /identity/v3/projects/b396c4245df1455bac98c4591a3f0772/users/88b91495fe4b4380972638e63523f195/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:48:58.032560 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4556d-893c-4aec-b097-95dc51fabab6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:58.032895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4556d-893c-4aec-b097-95dc51fabab6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:58.033199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4556d-893c-4aec-b097-95dc51fabab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:58.061467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-48e4556d-893c-4aec-b097-95dc51fabab6 None None] MFA Rules not processed for user `88b91495fe4b4380972638e63523f195`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:48:58.062708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-48e4556d-893c-4aec-b097-95dc51fabab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:58.072551 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:48:58.126558 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1289/2584] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:48:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:48:58.138686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e972a61-da01-495a-8e95-19d2996f22da admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:48:58.140524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3e972a61-da01-495a-8e95-19d2996f22da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:48:58.198830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e972a61-da01-495a-8e95-19d2996f22da admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:48:58.202218 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571e8c4-9b91-4bd8-b0a3-1306f46b2999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:48:58.202836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571e8c4-9b91-4bd8-b0a3-1306f46b2999 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:48:58.203120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571e8c4-9b91-4bd8-b0a3-1306f46b2999 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:48:58.203397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571e8c4-9b91-4bd8-b0a3-1306f46b2999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:48:58.212786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571e8c4-9b91-4bd8-b0a3-1306f46b2999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:48:58.214261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571e8c4-9b91-4bd8-b0a3-1306f46b2999 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:48:58.223436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1296/2585] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:48:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:08.130536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=22 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:08.219345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:11.430499 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4556d-893c-4aec-b097-95dc51fabab6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:11.431942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-48e4556d-893c-4aec-b097-95dc51fabab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:11.488502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4556d-893c-4aec-b097-95dc51fabab6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:11.492280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427ddfa-904f-44e5-9d4d-409692cef82e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:11.492927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427ddfa-904f-44e5-9d4d-409692cef82e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:11.493245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427ddfa-904f-44e5-9d4d-409692cef82e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:11.493527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427ddfa-904f-44e5-9d4d-409692cef82e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:11.495435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7427ddfa-904f-44e5-9d4d-409692cef82e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:11.558521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427ddfa-904f-44e5-9d4d-409692cef82e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:11.560043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427ddfa-904f-44e5-9d4d-409692cef82e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:11.569953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1290/2586] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:13.850206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571e8c4-9b91-4bd8-b0a3-1306f46b2999 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:13.853169 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8571e8c4-9b91-4bd8-b0a3-1306f46b2999 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:13.933237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571e8c4-9b91-4bd8-b0a3-1306f46b2999 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:13.939207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de43fef-767a-4a88-87d2-e99c30c14c38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:13.940194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de43fef-767a-4a88-87d2-e99c30c14c38 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:13.940633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de43fef-767a-4a88-87d2-e99c30c14c38 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:13.941337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de43fef-767a-4a88-87d2-e99c30c14c38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:13.943446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1de43fef-767a-4a88-87d2-e99c30c14c38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:13.957868 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de43fef-767a-4a88-87d2-e99c30c14c38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:13.959806 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de43fef-767a-4a88-87d2-e99c30c14c38 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:13.973591 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1297/2587] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:21.565582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=50 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:23.968464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=34 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:25.527298 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427ddfa-904f-44e5-9d4d-409692cef82e service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:25.598156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427ddfa-904f-44e5-9d4d-409692cef82e service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:25.603253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:25.603903 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:25.604213 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:25.604492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] PATH_INFO: `/v3/users/88b91495fe4b4380972638e63523f195` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:25.605655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88b91495fe4b4380972638e63523f195)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:25.607566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:25.874431 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88b91495fe4b4380972638e63523f195'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:25.875048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88b91495fe4b4380972638e63523f195'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:25.875651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88b91495fe4b4380972638e63523f195'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:25.887575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88b91495fe4b4380972638e63523f195'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:25.977362 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88b91495fe4b4380972638e63523f195'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:25.990991 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1291/2588] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:25 2022] DELETE /identity/v3/users/88b91495fe4b4380972638e63523f195 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:25.996263 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0ff2-b40c-4767-90f3-262d65fe0737 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:25.997070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0ff2-b40c-4767-90f3-262d65fe0737 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:25.997371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0ff2-b40c-4767-90f3-262d65fe0737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:26.016447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-edda0ff2-b40c-4767-90f3-262d65fe0737 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:26.017656 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-edda0ff2-b40c-4767-90f3-262d65fe0737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:26.076552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1298/2589] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:26.087081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:26.088515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:26.149400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b2b9e-6356-4b64-b35d-4ba94f3a8994 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:26.153074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aada5a-90ab-4e02-8717-fd0568db19bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:26.153712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aada5a-90ab-4e02-8717-fd0568db19bc service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:26.153985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aada5a-90ab-4e02-8717-fd0568db19bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:26.154263 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aada5a-90ab-4e02-8717-fd0568db19bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:26.163547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aada5a-90ab-4e02-8717-fd0568db19bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:26.165729 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aada5a-90ab-4e02-8717-fd0568db19bc service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:26.175889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1292/2590] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:49:26.277926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda0ff2-b40c-4767-90f3-262d65fe0737 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:26.279348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-edda0ff2-b40c-4767-90f3-262d65fe0737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:26.291356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda0ff2-b40c-4767-90f3-262d65fe0737 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:26.296064 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:26.296764 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:26.297046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:26.297324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] PATH_INFO: `/v3/users/51819962789845c8b56ed869eb28c54e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:26.298534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51819962789845c8b56ed869eb28c54e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:26.300643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:26.460784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51819962789845c8b56ed869eb28c54e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:26.461731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51819962789845c8b56ed869eb28c54e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:26.468892 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51819962789845c8b56ed869eb28c54e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:26.483013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51819962789845c8b56ed869eb28c54e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:26.553794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51819962789845c8b56ed869eb28c54e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:26.561487 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1299/2591] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:26 2022] DELETE /identity/v3/users/51819962789845c8b56ed869eb28c54e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:26.566160 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8620a0-d251-4749-babb-8c6825e9d661 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:26.566643 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8620a0-d251-4749-babb-8c6825e9d661 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:26.566795 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8620a0-d251-4749-babb-8c6825e9d661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:26.589158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dc8620a0-d251-4749-babb-8c6825e9d661 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:26.589333 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dc8620a0-d251-4749-babb-8c6825e9d661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:26.608230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:26.626975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:26.628571 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:26.679230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1293/2592] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:26.689832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8620a0-d251-4749-babb-8c6825e9d661 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:26.691334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dc8620a0-d251-4749-babb-8c6825e9d661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:26.708978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e516e-8be2-4553-b142-7bfeff141ce9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:26.714743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:26.715405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:26.715691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:26.715969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] PATH_INFO: `/v3/projects/b396c4245df1455bac98c4591a3f0772` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:26.717260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b396c4245df1455bac98c4591a3f0772)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:26.719204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:26.736025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:26.753752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8620a0-d251-4749-babb-8c6825e9d661 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:26.758252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4b2e7-76a4-49c4-b0b3-2ca8e705ed09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:26.763836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4b2e7-76a4-49c4-b0b3-2ca8e705ed09 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:26.763836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4b2e7-76a4-49c4-b0b3-2ca8e705ed09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:26.763836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4b2e7-76a4-49c4-b0b3-2ca8e705ed09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:26.768735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4b2e7-76a4-49c4-b0b3-2ca8e705ed09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:26.770280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4b2e7-76a4-49c4-b0b3-2ca8e705ed09 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:26.779711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1294/2593] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:49:26.841618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b396c4245df1455bac98c4591a3f0772', 'request_id': 'req-6d02e191-ccf1-4123-8d3b-c5057b7dc248'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:26.942201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b396c4245df1455bac98c4591a3f0772', 'request_id': 'req-6d02e191-ccf1-4123-8d3b-c5057b7dc248'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:26.949077 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b396c4245df1455bac98c4591a3f0772', 'request_id': 'req-6d02e191-ccf1-4123-8d3b-c5057b7dc248'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:26.961772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] The token cache is being invalidate because project b396c4245df1455bac98c4591a3f0772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:26.961876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:26.962516 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6d02e191-ccf1-4123-8d3b-c5057b7dc248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:26.964088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1300/2594] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:26 2022] DELETE /identity/v3/projects/b396c4245df1455bac98c4591a3f0772 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:26.969396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ad501-2146-469a-8a8a-e583bf562c5f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:26.969768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ad501-2146-469a-8a8a-e583bf562c5f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:26.970111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ad501-2146-469a-8a8a-e583bf562c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:26.993509 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-533ad501-2146-469a-8a8a-e583bf562c5f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:26.995193 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-533ad501-2146-469a-8a8a-e583bf562c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:27.072133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1295/2595] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:27.079876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9b505-5bda-4277-ac57-ffec4966d16d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:27.080423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9b505-5bda-4277-ac57-ffec4966d16d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:27.080537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9b505-5bda-4277-ac57-ffec4966d16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:27.100958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8ff9b505-5bda-4277-ac57-ffec4966d16d None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:27.102989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8ff9b505-5bda-4277-ac57-ffec4966d16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:27.112504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1301/2596] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:27.116636 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ad501-2146-469a-8a8a-e583bf562c5f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:27.127469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ad501-2146-469a-8a8a-e583bf562c5f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:27.133263 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137957f6-9911-49ee-952c-cf5c982d1054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:27.133960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137957f6-9911-49ee-952c-cf5c982d1054 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:27.134247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137957f6-9911-49ee-952c-cf5c982d1054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:27.134527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137957f6-9911-49ee-952c-cf5c982d1054 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:27.135230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137957f6-9911-49ee-952c-cf5c982d1054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:49:27.135538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137957f6-9911-49ee-952c-cf5c982d1054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:27.137182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137957f6-9911-49ee-952c-cf5c982d1054 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:27.157798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1296/2597] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:49:27.164696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9b505-5bda-4277-ac57-ffec4966d16d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:27.174316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9b505-5bda-4277-ac57-ffec4966d16d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:27.179231 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3083cd9-b4eb-464d-8267-af314db2651e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:27.180054 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3083cd9-b4eb-464d-8267-af314db2651e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:27.180338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3083cd9-b4eb-464d-8267-af314db2651e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:27.180612 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3083cd9-b4eb-464d-8267-af314db2651e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:27.181304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3083cd9-b4eb-464d-8267-af314db2651e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:27.183225 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3083cd9-b4eb-464d-8267-af314db2651e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:27.285726 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137957f6-9911-49ee-952c-cf5c982d1054 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:27.288693 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-137957f6-9911-49ee-952c-cf5c982d1054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:27.301868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137957f6-9911-49ee-952c-cf5c982d1054 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:27.307304 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:27.307950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:27.308273 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:27.308548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] PATH_INFO: `/v3/projects/23f6b9048c7f48e0bdc37ee1d19df177` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:27.309734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23f6b9048c7f48e0bdc37ee1d19df177)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:27.311615 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:27.326943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1302/2598] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:27 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:27.329086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3083cd9-b4eb-464d-8267-af314db2651e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:27.395549 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3083cd9-b4eb-464d-8267-af314db2651e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:27.400277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd61943-9502-4982-836a-b3ad471229a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:27.400904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd61943-9502-4982-836a-b3ad471229a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:27.401192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd61943-9502-4982-836a-b3ad471229a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:27.401476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd61943-9502-4982-836a-b3ad471229a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:27.402502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd61943-9502-4982-836a-b3ad471229a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:27.404175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd61943-9502-4982-836a-b3ad471229a4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:27.435465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23f6b9048c7f48e0bdc37ee1d19df177', 'request_id': 'req-3d50134c-36fc-48f9-8e6d-4126cd807f43'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:27.442946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23f6b9048c7f48e0bdc37ee1d19df177', 'request_id': 'req-3d50134c-36fc-48f9-8e6d-4126cd807f43'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:27.514812 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1303/2599] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:27 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:27.518820 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd61943-9502-4982-836a-b3ad471229a4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:27.528230 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd61943-9502-4982-836a-b3ad471229a4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:27.542393 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23f6b9048c7f48e0bdc37ee1d19df177', 'request_id': 'req-3d50134c-36fc-48f9-8e6d-4126cd807f43'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:27.553258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] The token cache is being invalidate because project 23f6b9048c7f48e0bdc37ee1d19df177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:27.553552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:27.554965 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:27.556731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1297/2600] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:27 2022] DELETE /identity/v3/projects/23f6b9048c7f48e0bdc37ee1d19df177 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:27.560812 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:27.562268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:27.586092 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f35da6-6fab-4d5a-a6c5-6b168730e846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:27.586754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f35da6-6fab-4d5a-a6c5-6b168730e846 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:27.587042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f35da6-6fab-4d5a-a6c5-6b168730e846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:27.587321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f35da6-6fab-4d5a-a6c5-6b168730e846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:27.588046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f35da6-6fab-4d5a-a6c5-6b168730e846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:27.589696 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f35da6-6fab-4d5a-a6c5-6b168730e846 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:27.599646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1304/2601] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:49:27.603708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f35da6-6fab-4d5a-a6c5-6b168730e846 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:27.606045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-02f35da6-6fab-4d5a-a6c5-6b168730e846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:27.618803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f35da6-6fab-4d5a-a6c5-6b168730e846 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:27.623490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698faf9-88cd-4452-9a5e-58f8be0db019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:27.624158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698faf9-88cd-4452-9a5e-58f8be0db019 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:27.624433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698faf9-88cd-4452-9a5e-58f8be0db019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:27.624725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698faf9-88cd-4452-9a5e-58f8be0db019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:27.625523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698faf9-88cd-4452-9a5e-58f8be0db019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:27.627136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698faf9-88cd-4452-9a5e-58f8be0db019 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:27.629545 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50134c-36fc-48f9-8e6d-4126cd807f43 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:27.635157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:27.635798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:27.636112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:27.636392 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] PATH_INFO: `/v3/users/ca12953e6a8041149ec3b27bf18d1491` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:27.637578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca12953e6a8041149ec3b27bf18d1491)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:27.637712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1305/2602] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:49:27.639601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:27.641890 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698faf9-88cd-4452-9a5e-58f8be0db019 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:27.651701 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698faf9-88cd-4452-9a5e-58f8be0db019 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:27.656422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280dda4-0e3b-49d7-883d-7aaaff91294d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:27.657132 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280dda4-0e3b-49d7-883d-7aaaff91294d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:27.657417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280dda4-0e3b-49d7-883d-7aaaff91294d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:27.657697 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280dda4-0e3b-49d7-883d-7aaaff91294d admin admin] PATH_INFO: `/v3/projects/175e7a36d1e34c199eeb030b88cfccc5/users/4c648d4c6f254868bb56cc6bc8134ec7/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:27.672477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280dda4-0e3b-49d7-883d-7aaaff91294d admin admin] RBAC: Authorizing `identity:create_grant(project_id=175e7a36d1e34c199eeb030b88cfccc5, user_id=4c648d4c6f254868bb56cc6bc8134ec7, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:27.679889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:27.680041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280dda4-0e3b-49d7-883d-7aaaff91294d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:27.781633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1306/2603] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:27 2022] PUT /identity/v3/projects/175e7a36d1e34c199eeb030b88cfccc5/users/4c648d4c6f254868bb56cc6bc8134ec7/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:27.793177 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dea288-41cd-4e1b-8928-eb8a954d2a87 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:27.793493 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dea288-41cd-4e1b-8928-eb8a954d2a87 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:27.793804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dea288-41cd-4e1b-8928-eb8a954d2a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:27.822549 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-29dea288-41cd-4e1b-8928-eb8a954d2a87 None None] MFA Rules not processed for user `4c648d4c6f254868bb56cc6bc8134ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:27.823741 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-29dea288-41cd-4e1b-8928-eb8a954d2a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:27.852944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca12953e6a8041149ec3b27bf18d1491'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:27.853858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca12953e6a8041149ec3b27bf18d1491'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:27.854362 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca12953e6a8041149ec3b27bf18d1491'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:27.866396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca12953e6a8041149ec3b27bf18d1491'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:27.889714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1307/2604] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:27.894129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dea288-41cd-4e1b-8928-eb8a954d2a87 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:27.895446 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-29dea288-41cd-4e1b-8928-eb8a954d2a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:27.931827 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca12953e6a8041149ec3b27bf18d1491'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:27.945066 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1298/2605] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:27 2022] DELETE /identity/v3/users/ca12953e6a8041149ec3b27bf18d1491 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:27.966035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dea288-41cd-4e1b-8928-eb8a954d2a87 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:28.025190 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6792830-803c-422c-9513-283f76ecf2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:28.026016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6792830-803c-422c-9513-283f76ecf2f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.026480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6792830-803c-422c-9513-283f76ecf2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.028420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:28.028903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6792830-803c-422c-9513-283f76ecf2f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.029633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6792830-803c-422c-9513-283f76ecf2f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:28.031632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6792830-803c-422c-9513-283f76ecf2f4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:28.161987 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1308/2606] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:27 2022] POST /identity/v3/projects => generated 375 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:28.163530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:28.174288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a4a2d-976b-43fb-8bdd-6d186edba093 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:28.240268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155fb6b-2c93-483f-9b36-51176229fed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:28.240922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155fb6b-2c93-483f-9b36-51176229fed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.241206 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155fb6b-2c93-483f-9b36-51176229fed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.241478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155fb6b-2c93-483f-9b36-51176229fed4 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.242224 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155fb6b-2c93-483f-9b36-51176229fed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:28.244214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155fb6b-2c93-483f-9b36-51176229fed4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:28.359031 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1299/2607] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:28 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:28.363039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6792830-803c-422c-9513-283f76ecf2f4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:28.373517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6792830-803c-422c-9513-283f76ecf2f4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:28.378278 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411e49b-9067-453b-9a12-b7bf7e2ce353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:28.378881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411e49b-9067-453b-9a12-b7bf7e2ce353 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.379164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411e49b-9067-453b-9a12-b7bf7e2ce353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.379440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411e49b-9067-453b-9a12-b7bf7e2ce353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.380160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c411e49b-9067-453b-9a12-b7bf7e2ce353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:28.381851 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c411e49b-9067-453b-9a12-b7bf7e2ce353 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:28.392360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1309/2608] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:28.396375 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155fb6b-2c93-483f-9b36-51176229fed4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:28.407429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155fb6b-2c93-483f-9b36-51176229fed4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:28.413472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5d818-2f60-44ca-8487-9f1f3984555e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:28.413472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5d818-2f60-44ca-8487-9f1f3984555e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.413472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5d818-2f60-44ca-8487-9f1f3984555e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.413472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5d818-2f60-44ca-8487-9f1f3984555e admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.414697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5d818-2f60-44ca-8487-9f1f3984555e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:28.415128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411e49b-9067-453b-9a12-b7bf7e2ce353 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:28.416361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5d818-2f60-44ca-8487-9f1f3984555e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:28.417308 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c411e49b-9067-453b-9a12-b7bf7e2ce353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:28.430138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1300/2609] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:28.431657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5d818-2f60-44ca-8487-9f1f3984555e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:28.439702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411e49b-9067-453b-9a12-b7bf7e2ce353 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:28.440172 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:28.440849 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.441135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.441395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5d818-2f60-44ca-8487-9f1f3984555e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:28.441531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] PATH_INFO: `/v3/projects/748d705bbef3445fbca130698621d7b7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.442617 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=748d705bbef3445fbca130698621d7b7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:28.444635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:28.447215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97b865-0ad5-4b15-90b9-cbc7fa01feec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:28.448146 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97b865-0ad5-4b15-90b9-cbc7fa01feec admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.448277 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97b865-0ad5-4b15-90b9-cbc7fa01feec admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.448547 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97b865-0ad5-4b15-90b9-cbc7fa01feec admin admin] PATH_INFO: `/v3/projects/f6142a3500bb4763999a7dc2d764ab91/users/68356e32d7f74fad879111fb563d8350/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.452650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:28.467113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97b865-0ad5-4b15-90b9-cbc7fa01feec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6142a3500bb4763999a7dc2d764ab91, user_id=68356e32d7f74fad879111fb563d8350, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:28.469922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97b865-0ad5-4b15-90b9-cbc7fa01feec admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:28.560209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1301/2610] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:28 2022] PUT /identity/v3/projects/f6142a3500bb4763999a7dc2d764ab91/users/68356e32d7f74fad879111fb563d8350/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:28.567498 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '748d705bbef3445fbca130698621d7b7', 'request_id': 'req-79f28d6f-07a0-4dd5-876f-1f124601f519'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:28.570775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adb86b-39b7-462b-8980-5c0e7d153b3d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.571067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adb86b-39b7-462b-8980-5c0e7d153b3d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.571342 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adb86b-39b7-462b-8980-5c0e7d153b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.599290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-82adb86b-39b7-462b-8980-5c0e7d153b3d None None] MFA Rules not processed for user `68356e32d7f74fad879111fb563d8350`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:28.600552 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-82adb86b-39b7-462b-8980-5c0e7d153b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:28.645165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '748d705bbef3445fbca130698621d7b7', 'request_id': 'req-79f28d6f-07a0-4dd5-876f-1f124601f519'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:28.652037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '748d705bbef3445fbca130698621d7b7', 'request_id': 'req-79f28d6f-07a0-4dd5-876f-1f124601f519'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:28.663030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] The token cache is being invalidate because project 748d705bbef3445fbca130698621d7b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:28.663348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:28.664786 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-79f28d6f-07a0-4dd5-876f-1f124601f519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:28.666659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1310/2611] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:28 2022] DELETE /identity/v3/projects/748d705bbef3445fbca130698621d7b7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:28.670157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1302/2612] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:28.671472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086f040-96f4-4afd-b2a8-7139a3e41553 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.671765 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086f040-96f4-4afd-b2a8-7139a3e41553 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.672181 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086f040-96f4-4afd-b2a8-7139a3e41553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.674345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adb86b-39b7-462b-8980-5c0e7d153b3d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:28.696490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4086f040-96f4-4afd-b2a8-7139a3e41553 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:28.696490 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4086f040-96f4-4afd-b2a8-7139a3e41553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:28.727461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:28.740612 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adb86b-39b7-462b-8980-5c0e7d153b3d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:28.745941 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c5494-13c9-4935-8d4f-1ef147ce66df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:28.746559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c5494-13c9-4935-8d4f-1ef147ce66df admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.746876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c5494-13c9-4935-8d4f-1ef147ce66df admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.747153 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c5494-13c9-4935-8d4f-1ef147ce66df admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.747816 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c5494-13c9-4935-8d4f-1ef147ce66df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:28.749793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c5494-13c9-4935-8d4f-1ef147ce66df admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:28.760995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1311/2613] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 04 07:49:28.768975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f52bb-1323-48d1-97bf-495b577d0691 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.772550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f52bb-1323-48d1-97bf-495b577d0691 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.772550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f52bb-1323-48d1-97bf-495b577d0691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.792700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f03f52bb-1323-48d1-97bf-495b577d0691 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:28.793917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f03f52bb-1323-48d1-97bf-495b577d0691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:28.803042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1312/2614] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:28.807840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f52bb-1323-48d1-97bf-495b577d0691 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:28.818755 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f52bb-1323-48d1-97bf-495b577d0691 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:28.824779 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fa70e-5194-442d-9033-bab7a5edb18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:28.825398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fa70e-5194-442d-9033-bab7a5edb18e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.825684 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fa70e-5194-442d-9033-bab7a5edb18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.825961 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fa70e-5194-442d-9033-bab7a5edb18e admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.826658 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fa70e-5194-442d-9033-bab7a5edb18e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:49:28.826966 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fa70e-5194-442d-9033-bab7a5edb18e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:28.829036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fa70e-5194-442d-9033-bab7a5edb18e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:28.840875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1313/2615] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:49:28.845103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fa70e-5194-442d-9033-bab7a5edb18e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:28.854740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fa70e-5194-442d-9033-bab7a5edb18e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:28.861724 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472832d8-16ea-4b72-a22b-a6eac2ffd552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:28.862133 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472832d8-16ea-4b72-a22b-a6eac2ffd552 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.862420 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472832d8-16ea-4b72-a22b-a6eac2ffd552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.862698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472832d8-16ea-4b72-a22b-a6eac2ffd552 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.863435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472832d8-16ea-4b72-a22b-a6eac2ffd552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:28.865385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472832d8-16ea-4b72-a22b-a6eac2ffd552 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:28.872406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1303/2616] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:28 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:28.873284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c5494-13c9-4935-8d4f-1ef147ce66df admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:28.876483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e19c5494-13c9-4935-8d4f-1ef147ce66df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:28.905832 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:28.942712 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c5494-13c9-4935-8d4f-1ef147ce66df admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:28.951871 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1314/2617] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:28 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:28.952107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03135be-e247-4c22-ac4a-b29f1757144a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:28.952637 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03135be-e247-4c22-ac4a-b29f1757144a admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:28.952921 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03135be-e247-4c22-ac4a-b29f1757144a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:28.953199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03135be-e247-4c22-ac4a-b29f1757144a admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:28.954119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03135be-e247-4c22-ac4a-b29f1757144a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:28.956248 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03135be-e247-4c22-ac4a-b29f1757144a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:28.960731 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472832d8-16ea-4b72-a22b-a6eac2ffd552 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:28.980910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:29.022672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472832d8-16ea-4b72-a22b-a6eac2ffd552 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.027593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14694ef0-8e27-4197-98c2-4b43f423a95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.028324 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14694ef0-8e27-4197-98c2-4b43f423a95f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.028675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14694ef0-8e27-4197-98c2-4b43f423a95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.029025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14694ef0-8e27-4197-98c2-4b43f423a95f admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.029756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14694ef0-8e27-4197-98c2-4b43f423a95f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.034937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14694ef0-8e27-4197-98c2-4b43f423a95f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.040540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1304/2618] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:28 2022] POST /identity/v3/users => generated 295 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:29.043451 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03135be-e247-4c22-ac4a-b29f1757144a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.056022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03135be-e247-4c22-ac4a-b29f1757144a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.060739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b67384-2cf0-4285-98df-c5fcfd3ecaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.061357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b67384-2cf0-4285-98df-c5fcfd3ecaaa admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.063512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b67384-2cf0-4285-98df-c5fcfd3ecaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.063512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b67384-2cf0-4285-98df-c5fcfd3ecaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.063512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b67384-2cf0-4285-98df-c5fcfd3ecaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.064318 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b67384-2cf0-4285-98df-c5fcfd3ecaaa admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.075807 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1305/2619] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:49:29.080374 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b67384-2cf0-4285-98df-c5fcfd3ecaaa admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.094652 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b67384-2cf0-4285-98df-c5fcfd3ecaaa admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.099246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14a6e-105b-4545-99c9-6be4e783e000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.100024 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14a6e-105b-4545-99c9-6be4e783e000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.100313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14a6e-105b-4545-99c9-6be4e783e000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.100587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14a6e-105b-4545-99c9-6be4e783e000 admin admin] PATH_INFO: `/v3/projects/2d90076372974667a1cc5499a472411c/users/e56c688c8c9a4067acbd030137131bc6/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.115955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14a6e-105b-4545-99c9-6be4e783e000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d90076372974667a1cc5499a472411c, user_id=e56c688c8c9a4067acbd030137131bc6, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.119236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14a6e-105b-4545-99c9-6be4e783e000 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.121004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1315/2620] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:28 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:29.126920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14694ef0-8e27-4197-98c2-4b43f423a95f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.135423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14694ef0-8e27-4197-98c2-4b43f423a95f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.140409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb9e2c-b03e-472f-a5a7-5cb1675a3045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.141051 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb9e2c-b03e-472f-a5a7-5cb1675a3045 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.141550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb9e2c-b03e-472f-a5a7-5cb1675a3045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.142006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb9e2c-b03e-472f-a5a7-5cb1675a3045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.143070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb9e2c-b03e-472f-a5a7-5cb1675a3045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.144771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb9e2c-b03e-472f-a5a7-5cb1675a3045 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.155687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1316/2621] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:29.159438 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb9e2c-b03e-472f-a5a7-5cb1675a3045 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.169733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb9e2c-b03e-472f-a5a7-5cb1675a3045 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.174394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd6d04-30bf-4c13-a536-dba5fdd4a972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.174957 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd6d04-30bf-4c13-a536-dba5fdd4a972 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.175241 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd6d04-30bf-4c13-a536-dba5fdd4a972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.175515 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd6d04-30bf-4c13-a536-dba5fdd4a972 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.176203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd6d04-30bf-4c13-a536-dba5fdd4a972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.177910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd6d04-30bf-4c13-a536-dba5fdd4a972 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.188699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1317/2622] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:49:29.192729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd6d04-30bf-4c13-a536-dba5fdd4a972 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.203038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd6d04-30bf-4c13-a536-dba5fdd4a972 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.207972 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fdf37-d680-411b-8e16-bc0b2d93694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.208670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fdf37-d680-411b-8e16-bc0b2d93694c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.208975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fdf37-d680-411b-8e16-bc0b2d93694c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.209236 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fdf37-d680-411b-8e16-bc0b2d93694c admin admin] PATH_INFO: `/v3/projects/73058b10749c4432987a660ffad9a838/users/ac1f78036e0b49689af602b45fa312ab/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.213402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1306/2623] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:29 2022] PUT /identity/v3/projects/2d90076372974667a1cc5499a472411c/users/e56c688c8c9a4067acbd030137131bc6/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:29.216961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14a6e-105b-4545-99c9-6be4e783e000 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.225108 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fdf37-d680-411b-8e16-bc0b2d93694c admin admin] RBAC: Authorizing `identity:create_grant(project_id=73058b10749c4432987a660ffad9a838, user_id=ac1f78036e0b49689af602b45fa312ab, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.227810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fdf37-d680-411b-8e16-bc0b2d93694c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.280182 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14a6e-105b-4545-99c9-6be4e783e000 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.285034 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5f053-cc5d-4606-b00b-8abef9b1ded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.285651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5f053-cc5d-4606-b00b-8abef9b1ded6 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.286041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5f053-cc5d-4606-b00b-8abef9b1ded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.286244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5f053-cc5d-4606-b00b-8abef9b1ded6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.286991 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5f053-cc5d-4606-b00b-8abef9b1ded6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.289335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5f053-cc5d-4606-b00b-8abef9b1ded6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.299290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1307/2624] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:49:29.303736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5f053-cc5d-4606-b00b-8abef9b1ded6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.313313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5f053-cc5d-4606-b00b-8abef9b1ded6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.317849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f305fbf-5e3c-464a-8290-5284d874221a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.318495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305fbf-5e3c-464a-8290-5284d874221a admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.318775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305fbf-5e3c-464a-8290-5284d874221a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.319050 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305fbf-5e3c-464a-8290-5284d874221a admin admin] PATH_INFO: `/v3/projects/2d90076372974667a1cc5499a472411c/users/e56c688c8c9a4067acbd030137131bc6/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.322103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f305fbf-5e3c-464a-8290-5284d874221a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d90076372974667a1cc5499a472411c, user_id=e56c688c8c9a4067acbd030137131bc6, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.324661 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1318/2625] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:29 2022] PUT /identity/v3/projects/73058b10749c4432987a660ffad9a838/users/ac1f78036e0b49689af602b45fa312ab/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:29.324905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f305fbf-5e3c-464a-8290-5284d874221a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.337761 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1308/2626] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:29 2022] PUT /identity/v3/projects/2d90076372974667a1cc5499a472411c/users/e56c688c8c9a4067acbd030137131bc6/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:29.339518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1a0b0-a553-4a71-b0bb-343861ff785f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.339802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1a0b0-a553-4a71-b0bb-343861ff785f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.340106 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1a0b0-a553-4a71-b0bb-343861ff785f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.385285 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d8e1a0b0-a553-4a71-b0bb-343861ff785f None None] MFA Rules not processed for user `ac1f78036e0b49689af602b45fa312ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:29.386563 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d8e1a0b0-a553-4a71-b0bb-343861ff785f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:29.407754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a98237-8471-4c9f-b839-085fdb91cf2f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.408090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a98237-8471-4c9f-b839-085fdb91cf2f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.408367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a98237-8471-4c9f-b839-085fdb91cf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.437403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-77a98237-8471-4c9f-b839-085fdb91cf2f None None] MFA Rules not processed for user `e56c688c8c9a4067acbd030137131bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:29.438731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-77a98237-8471-4c9f-b839-085fdb91cf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:29.463357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1319/2627] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:29.463357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1a0b0-a553-4a71-b0bb-343861ff785f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.492259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:29.514332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:29.516063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1309/2628] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:29.528517 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a98237-8471-4c9f-b839-085fdb91cf2f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.529752 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1a0b0-a553-4a71-b0bb-343861ff785f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.530090 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-77a98237-8471-4c9f-b839-085fdb91cf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:29.534814 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845179d-c093-47c7-8306-6f8afdab16c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.535427 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845179d-c093-47c7-8306-6f8afdab16c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.535714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845179d-c093-47c7-8306-6f8afdab16c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.536004 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845179d-c093-47c7-8306-6f8afdab16c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.536764 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845179d-c093-47c7-8306-6f8afdab16c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.538715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845179d-c093-47c7-8306-6f8afdab16c7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.589245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a98237-8471-4c9f-b839-085fdb91cf2f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.593108 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b2a24-f24b-4ca6-bcf4-00fa9c29218b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.595321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b2a24-f24b-4ca6-bcf4-00fa9c29218b service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.595321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b2a24-f24b-4ca6-bcf4-00fa9c29218b service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.595321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b2a24-f24b-4ca6-bcf4-00fa9c29218b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.596367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-960b2a24-f24b-4ca6-bcf4-00fa9c29218b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:29.624015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1320/2629] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:29 2022] POST /identity/v3/projects => generated 369 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:29.627691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845179d-c093-47c7-8306-6f8afdab16c7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.656986 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b2a24-f24b-4ca6-bcf4-00fa9c29218b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.658425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b2a24-f24b-4ca6-bcf4-00fa9c29218b service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.667363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1310/2630] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:49:29.690494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845179d-c093-47c7-8306-6f8afdab16c7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.695306 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c185835-1024-4b3e-b49a-44536973e937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.696429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c185835-1024-4b3e-b49a-44536973e937 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.696719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c185835-1024-4b3e-b49a-44536973e937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.696973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c185835-1024-4b3e-b49a-44536973e937 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.697622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c185835-1024-4b3e-b49a-44536973e937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.699512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c185835-1024-4b3e-b49a-44536973e937 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.810038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1321/2631] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:29 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:29.814091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b2a24-f24b-4ca6-bcf4-00fa9c29218b service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.825572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b2a24-f24b-4ca6-bcf4-00fa9c29218b service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.830516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f60b4-4192-4291-9314-d567befe35b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.831202 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f60b4-4192-4291-9314-d567befe35b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.831489 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f60b4-4192-4291-9314-d567befe35b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.831769 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f60b4-4192-4291-9314-d567befe35b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.832537 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f60b4-4192-4291-9314-d567befe35b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.834123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f60b4-4192-4291-9314-d567befe35b9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:29.844435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1311/2632] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:29.850061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c185835-1024-4b3e-b49a-44536973e937 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:29.860207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c185835-1024-4b3e-b49a-44536973e937 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:29.864959 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b407626-62fd-46d4-b487-3b98cec32ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:29.865614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b407626-62fd-46d4-b487-3b98cec32ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:29.865896 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b407626-62fd-46d4-b487-3b98cec32ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:29.866216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b407626-62fd-46d4-b487-3b98cec32ba5 admin admin] PATH_INFO: `/v3/projects/38d87d74979544f7aaae335eecc36298/users/880a8383f85246419a8ea8d4a11f9191/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:29.885825 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b407626-62fd-46d4-b487-3b98cec32ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d87d74979544f7aaae335eecc36298, user_id=880a8383f85246419a8ea8d4a11f9191, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:29.888714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b407626-62fd-46d4-b487-3b98cec32ba5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:30.001165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1322/2633] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:29 2022] PUT /identity/v3/projects/38d87d74979544f7aaae335eecc36298/users/880a8383f85246419a8ea8d4a11f9191/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:30.005153 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f60b4-4192-4291-9314-d567befe35b9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:30.015421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f60b4-4192-4291-9314-d567befe35b9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:30.071254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523e8c4-e743-4a42-8cd7-ca6f6404c1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:30.071863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523e8c4-e743-4a42-8cd7-ca6f6404c1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:30.072178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523e8c4-e743-4a42-8cd7-ca6f6404c1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:30.072453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523e8c4-e743-4a42-8cd7-ca6f6404c1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:30.073169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523e8c4-e743-4a42-8cd7-ca6f6404c1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:30.074753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523e8c4-e743-4a42-8cd7-ca6f6404c1b1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:30.084890 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1312/2634] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:30.089377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b407626-62fd-46d4-b487-3b98cec32ba5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:30.100810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b407626-62fd-46d4-b487-3b98cec32ba5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:30.106454 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c836a3-9028-45f0-bc95-0e8981ab1240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:30.107116 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c836a3-9028-45f0-bc95-0e8981ab1240 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:30.107414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c836a3-9028-45f0-bc95-0e8981ab1240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:30.107670 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c836a3-9028-45f0-bc95-0e8981ab1240 admin admin] PATH_INFO: `/v3/projects/38d87d74979544f7aaae335eecc36298/users/880a8383f85246419a8ea8d4a11f9191/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:30.109662 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c836a3-9028-45f0-bc95-0e8981ab1240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d87d74979544f7aaae335eecc36298, user_id=880a8383f85246419a8ea8d4a11f9191, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:30.112333 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c836a3-9028-45f0-bc95-0e8981ab1240 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:30.125256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1323/2635] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:30 2022] PUT /identity/v3/projects/38d87d74979544f7aaae335eecc36298/users/880a8383f85246419a8ea8d4a11f9191/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:30.135771 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09c097-ebc9-4662-b64c-0a15ea662656 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:30.136167 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09c097-ebc9-4662-b64c-0a15ea662656 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:30.136539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09c097-ebc9-4662-b64c-0a15ea662656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:30.170869 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6c09c097-ebc9-4662-b64c-0a15ea662656 None None] MFA Rules not processed for user `880a8383f85246419a8ea8d4a11f9191`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:30.170869 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6c09c097-ebc9-4662-b64c-0a15ea662656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:30.196262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:30.197279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c836a3-9028-45f0-bc95-0e8981ab1240 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:30.199339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b1c836a3-9028-45f0-bc95-0e8981ab1240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:30.260418 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1313/2636] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:30.267700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c836a3-9028-45f0-bc95-0e8981ab1240 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:30.270102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09c097-ebc9-4662-b64c-0a15ea662656 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:30.273709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2c61c-06a4-4c1a-bed7-9e71381c408d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:30.273709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2c61c-06a4-4c1a-bed7-9e71381c408d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:30.273709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2c61c-06a4-4c1a-bed7-9e71381c408d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:30.273709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2c61c-06a4-4c1a-bed7-9e71381c408d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:30.274620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6c09c097-ebc9-4662-b64c-0a15ea662656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:30.275653 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a0a2c61c-06a4-4c1a-bed7-9e71381c408d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:30.326667 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:30.347243 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09c097-ebc9-4662-b64c-0a15ea662656 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:30.349712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2c61c-06a4-4c1a-bed7-9e71381c408d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:30.350078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2c61c-06a4-4c1a-bed7-9e71381c408d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:30.352367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efafa9e-fd1d-423a-87f4-5cea53a412c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:30.352990 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efafa9e-fd1d-423a-87f4-5cea53a412c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:30.353274 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efafa9e-fd1d-423a-87f4-5cea53a412c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:30.353546 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efafa9e-fd1d-423a-87f4-5cea53a412c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:30.363172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efafa9e-fd1d-423a-87f4-5cea53a412c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:30.364777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efafa9e-fd1d-423a-87f4-5cea53a412c5 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:30.369499 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1324/2637] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:30.377023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1314/2638] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:49:31.637757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2c61c-06a4-4c1a-bed7-9e71381c408d service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:31.647646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2c61c-06a4-4c1a-bed7-9e71381c408d service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:31.651541 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616fe0e-6085-4624-b877-4062178f2c5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:31.652188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616fe0e-6085-4624-b877-4062178f2c5b service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:31.652475 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616fe0e-6085-4624-b877-4062178f2c5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:31.652749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616fe0e-6085-4624-b877-4062178f2c5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:31.654528 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3616fe0e-6085-4624-b877-4062178f2c5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:31.724980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616fe0e-6085-4624-b877-4062178f2c5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:31.727150 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616fe0e-6085-4624-b877-4062178f2c5b service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:31.737659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1325/2639] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:33.662620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efafa9e-fd1d-423a-87f4-5cea53a412c5 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:33.672095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efafa9e-fd1d-423a-87f4-5cea53a412c5 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:33.694102 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107a901-ff4b-44b0-9905-f4c304ef59a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:33.695724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107a901-ff4b-44b0-9905-f4c304ef59a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:33.696619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107a901-ff4b-44b0-9905-f4c304ef59a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:33.697992 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107a901-ff4b-44b0-9905-f4c304ef59a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:33.716073 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616fe0e-6085-4624-b877-4062178f2c5b service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:33.728144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616fe0e-6085-4624-b877-4062178f2c5b service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:33.737714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:33.744883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:33.744883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:33.744883 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] PATH_INFO: `/v3/users/ac1f78036e0b49689af602b45fa312ab` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:33.752833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1f78036e0b49689af602b45fa312ab)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:33.754819 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:33.800021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107a901-ff4b-44b0-9905-f4c304ef59a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:33.802750 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107a901-ff4b-44b0-9905-f4c304ef59a1 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:33.820041 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1315/2640] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:34.008983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1f78036e0b49689af602b45fa312ab'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:34.012403 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1f78036e0b49689af602b45fa312ab'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:34.045562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1f78036e0b49689af602b45fa312ab'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:34.069063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1f78036e0b49689af602b45fa312ab'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:34.142335 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1f78036e0b49689af602b45fa312ab'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:34.150774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1326/2641] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:33 2022] DELETE /identity/v3/users/ac1f78036e0b49689af602b45fa312ab => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:34.154786 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec34993-a3c3-433d-a315-b8755ec0566b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:34.155083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec34993-a3c3-433d-a315-b8755ec0566b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:34.155364 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec34993-a3c3-433d-a315-b8755ec0566b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:34.175469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0ec34993-a3c3-433d-a315-b8755ec0566b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:34.176719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0ec34993-a3c3-433d-a315-b8755ec0566b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:34.254429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:34.289482 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1316/2642] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:34.299614 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:34.301310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:34.370567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:34.374466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658edae-2c62-4fdb-afa0-8e552bc12d17 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:34.378075 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb4ab9-df18-45f0-af7b-2bde1f1819dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:34.378695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb4ab9-df18-45f0-af7b-2bde1f1819dc service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:34.378978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb4ab9-df18-45f0-af7b-2bde1f1819dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:34.379280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb4ab9-df18-45f0-af7b-2bde1f1819dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:34.389238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb4ab9-df18-45f0-af7b-2bde1f1819dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:34.390678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb4ab9-df18-45f0-af7b-2bde1f1819dc service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:34.400937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1327/2643] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:34.413184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec34993-a3c3-433d-a315-b8755ec0566b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:34.415038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0ec34993-a3c3-433d-a315-b8755ec0566b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:34.428889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec34993-a3c3-433d-a315-b8755ec0566b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:34.434867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:34.435626 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:34.436067 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:34.436606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] PATH_INFO: `/v3/users/e5fe46987d5942b3a0ec5495aa0040ac` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:34.437915 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5fe46987d5942b3a0ec5495aa0040ac)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:34.439906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:34.699281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5fe46987d5942b3a0ec5495aa0040ac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:34.700696 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5fe46987d5942b3a0ec5495aa0040ac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:34.701882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5fe46987d5942b3a0ec5495aa0040ac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:34.722388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5fe46987d5942b3a0ec5495aa0040ac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:34.839487 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5fe46987d5942b3a0ec5495aa0040ac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:34.853836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1317/2644] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:34 2022] DELETE /identity/v3/users/e5fe46987d5942b3a0ec5495aa0040ac => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:34.954018 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb4ab9-df18-45f0-af7b-2bde1f1819dc service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:34.957716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9dcb4ab9-df18-45f0-af7b-2bde1f1819dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:34.969895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb4ab9-df18-45f0-af7b-2bde1f1819dc service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:35.050562 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:35.051219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:35.051505 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:35.051778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] PATH_INFO: `/v3/projects/73058b10749c4432987a660ffad9a838` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:35.053012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73058b10749c4432987a660ffad9a838)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:35.055283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:35.177686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73058b10749c4432987a660ffad9a838', 'request_id': 'req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.191428 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:35.193015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:35.211873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157277a6-6454-4cf0-a342-5e123c877450 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:35.274020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73058b10749c4432987a660ffad9a838', 'request_id': 'req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.275453 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:35.276111 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:35.276397 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:35.276913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] PATH_INFO: `/v3/projects/ba45e1e6970248b192b9d665d4e45d1f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:35.277951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba45e1e6970248b192b9d665d4e45d1f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:35.280107 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:35.281622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73058b10749c4432987a660ffad9a838', 'request_id': 'req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.296148 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] The token cache is being invalidate because project 73058b10749c4432987a660ffad9a838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:35.296704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.297336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.302489 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1328/2645] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:34 2022] DELETE /identity/v3/projects/73058b10749c4432987a660ffad9a838 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:35.303920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:35.306124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:35.318558 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d6cc1-bae8-4b8a-8cfc-0af528b53f19 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:35.323405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:35.324074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:35.324357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:35.324633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] PATH_INFO: `/v3/users/880a8383f85246419a8ea8d4a11f9191` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:35.325803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=880a8383f85246419a8ea8d4a11f9191)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:35.327719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:35.469826 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba45e1e6970248b192b9d665d4e45d1f', 'request_id': 'req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.476858 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba45e1e6970248b192b9d665d4e45d1f', 'request_id': 'req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.541860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba45e1e6970248b192b9d665d4e45d1f', 'request_id': 'req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.544152 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:35.553174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] The token cache is being invalidate because project ba45e1e6970248b192b9d665d4e45d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:35.553485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.554144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cb3cca9c-31d1-4816-a699-ce50fa3c8f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.555639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1318/2646] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:35 2022] DELETE /identity/v3/projects/ba45e1e6970248b192b9d665d4e45d1f => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:35.560417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58f91b-e681-4c47-85fa-083c61b8ca59 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:35.560711 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58f91b-e681-4c47-85fa-083c61b8ca59 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:35.560992 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58f91b-e681-4c47-85fa-083c61b8ca59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:35.581573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ef58f91b-e681-4c47-85fa-083c61b8ca59 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:35.582736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ef58f91b-e681-4c47-85fa-083c61b8ca59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:35.586481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '880a8383f85246419a8ea8d4a11f9191'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.587142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '880a8383f85246419a8ea8d4a11f9191'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.594642 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '880a8383f85246419a8ea8d4a11f9191'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.607801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '880a8383f85246419a8ea8d4a11f9191'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.610382 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:35.643051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1319/2647] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:35.651301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e82eb-40ef-4671-9192-d6a302c8e276 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:35.651585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e82eb-40ef-4671-9192-d6a302c8e276 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:35.651860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e82eb-40ef-4671-9192-d6a302c8e276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:35.671244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-973e82eb-40ef-4671-9192-d6a302c8e276 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:35.673635 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-973e82eb-40ef-4671-9192-d6a302c8e276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:35.692398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1320/2648] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:35.696677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e82eb-40ef-4671-9192-d6a302c8e276 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:35.706305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e82eb-40ef-4671-9192-d6a302c8e276 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:35.710946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2914fa-b34b-4809-8082-ab20049569e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:35.711564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2914fa-b34b-4809-8082-ab20049569e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:35.711844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2914fa-b34b-4809-8082-ab20049569e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:35.712169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2914fa-b34b-4809-8082-ab20049569e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:35.719504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2914fa-b34b-4809-8082-ab20049569e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:49:35.719504 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2914fa-b34b-4809-8082-ab20049569e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:35.720010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '880a8383f85246419a8ea8d4a11f9191'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:35.720148 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2914fa-b34b-4809-8082-ab20049569e9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:35.723124 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1329/2649] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:35 2022] DELETE /identity/v3/users/880a8383f85246419a8ea8d4a11f9191 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:35.737885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1321/2650] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:49:35.739001 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:35.803408 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9415e-f3dc-4820-8fa2-c6be80c2b962 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:35.808447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08367e53-b6ab-4f80-a569-574b3c460c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:35.809067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08367e53-b6ab-4f80-a569-574b3c460c6a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:35.809346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08367e53-b6ab-4f80-a569-574b3c460c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:35.811461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08367e53-b6ab-4f80-a569-574b3c460c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:35.811461 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08367e53-b6ab-4f80-a569-574b3c460c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:35.812259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08367e53-b6ab-4f80-a569-574b3c460c6a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:35.956081 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1330/2651] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:35 2022] POST /identity/v3/projects => generated 385 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:35.960964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2914fa-b34b-4809-8082-ab20049569e9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:35.972165 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2914fa-b34b-4809-8082-ab20049569e9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:36.035721 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295f49b-d19d-4dc6-a0e0-9f718cf71dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:36.036395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295f49b-d19d-4dc6-a0e0-9f718cf71dce admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.038821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295f49b-d19d-4dc6-a0e0-9f718cf71dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.038821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295f49b-d19d-4dc6-a0e0-9f718cf71dce admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.038821 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295f49b-d19d-4dc6-a0e0-9f718cf71dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:36.039754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295f49b-d19d-4dc6-a0e0-9f718cf71dce admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:36.120283 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1322/2652] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:35 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:36.124512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08367e53-b6ab-4f80-a569-574b3c460c6a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:36.135506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08367e53-b6ab-4f80-a569-574b3c460c6a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:36.140268 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3ecdc-0380-4164-97c3-5f55ad4371ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:36.140884 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3ecdc-0380-4164-97c3-5f55ad4371ca admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.141168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3ecdc-0380-4164-97c3-5f55ad4371ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.141447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3ecdc-0380-4164-97c3-5f55ad4371ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.142141 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3ecdc-0380-4164-97c3-5f55ad4371ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:36.143734 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3ecdc-0380-4164-97c3-5f55ad4371ca admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:36.155831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1331/2653] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:36.159850 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295f49b-d19d-4dc6-a0e0-9f718cf71dce admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:36.169937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295f49b-d19d-4dc6-a0e0-9f718cf71dce admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:36.174421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c09a9-a134-4a2f-9df7-16df7e4cc93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:36.175069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c09a9-a134-4a2f-9df7-16df7e4cc93d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.175350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c09a9-a134-4a2f-9df7-16df7e4cc93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.175625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c09a9-a134-4a2f-9df7-16df7e4cc93d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.176345 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c09a9-a134-4a2f-9df7-16df7e4cc93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:36.177961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c09a9-a134-4a2f-9df7-16df7e4cc93d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:36.192420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1323/2654] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:36.196656 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3ecdc-0380-4164-97c3-5f55ad4371ca admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:36.207682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3ecdc-0380-4164-97c3-5f55ad4371ca admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:36.212646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0823ebe-6d89-4340-a434-d1635e80ce04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:36.214045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823ebe-6d89-4340-a434-d1635e80ce04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.214520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823ebe-6d89-4340-a434-d1635e80ce04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.215104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823ebe-6d89-4340-a434-d1635e80ce04 admin admin] PATH_INFO: `/v3/projects/ebc009a20a10430583f8a0a52d8f6903/users/bd535571fee44e17ba71c0f2d75b0aa7/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.230673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0823ebe-6d89-4340-a434-d1635e80ce04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc009a20a10430583f8a0a52d8f6903, user_id=bd535571fee44e17ba71c0f2d75b0aa7, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:36.232901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0823ebe-6d89-4340-a434-d1635e80ce04 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:36.259702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c09a9-a134-4a2f-9df7-16df7e4cc93d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:36.261373 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-433c09a9-a134-4a2f-9df7-16df7e4cc93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:36.279255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c09a9-a134-4a2f-9df7-16df7e4cc93d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:36.284763 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:36.285501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.285849 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.286300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] PATH_INFO: `/v3/projects/38d87d74979544f7aaae335eecc36298` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.287622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d87d74979544f7aaae335eecc36298)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:36.289899 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:36.353955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1332/2655] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:36 2022] PUT /identity/v3/projects/ebc009a20a10430583f8a0a52d8f6903/users/bd535571fee44e17ba71c0f2d75b0aa7/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:36.364345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984d409-e829-4dc3-b121-74dbaea4ebd0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.364631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984d409-e829-4dc3-b121-74dbaea4ebd0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.364913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984d409-e829-4dc3-b121-74dbaea4ebd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.394325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2984d409-e829-4dc3-b121-74dbaea4ebd0 None None] MFA Rules not processed for user `bd535571fee44e17ba71c0f2d75b0aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:36.395514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2984d409-e829-4dc3-b121-74dbaea4ebd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:36.452661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:36.456029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d87d74979544f7aaae335eecc36298', 'request_id': 'req-4c66a3f3-a9f1-40bb-85da-71096244af7a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:36.461907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1333/2656] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:36.463475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d87d74979544f7aaae335eecc36298', 'request_id': 'req-4c66a3f3-a9f1-40bb-85da-71096244af7a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:36.472565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984d409-e829-4dc3-b121-74dbaea4ebd0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:36.473956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2984d409-e829-4dc3-b121-74dbaea4ebd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:36.507915 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:36.545254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984d409-e829-4dc3-b121-74dbaea4ebd0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:36.605579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d87d74979544f7aaae335eecc36298', 'request_id': 'req-4c66a3f3-a9f1-40bb-85da-71096244af7a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:36.607517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a2ade-5e19-4ca2-837f-ce40c71cfb35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:36.608183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a2ade-5e19-4ca2-837f-ce40c71cfb35 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.608479 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a2ade-5e19-4ca2-837f-ce40c71cfb35 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.608738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a2ade-5e19-4ca2-837f-ce40c71cfb35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.616726 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] The token cache is being invalidate because project 38d87d74979544f7aaae335eecc36298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:36.617017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:36.617697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c66a3f3-a9f1-40bb-85da-71096244af7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:36.619238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1324/2657] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:36 2022] DELETE /identity/v3/projects/38d87d74979544f7aaae335eecc36298 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:36.624546 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c795bb-ff2f-447d-b273-a485b7e1667f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.624838 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c795bb-ff2f-447d-b273-a485b7e1667f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.625120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c795bb-ff2f-447d-b273-a485b7e1667f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.645089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b9c795bb-ff2f-447d-b273-a485b7e1667f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:36.646317 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b9c795bb-ff2f-447d-b273-a485b7e1667f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:36.667211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a2ade-5e19-4ca2-837f-ce40c71cfb35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:36.668877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a2ade-5e19-4ca2-837f-ce40c71cfb35 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:36.678643 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1334/2658] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:49:36.707175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1325/2659] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:36.716008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec503d3f-66c5-42fa-8ee1-1eb67cb7da22 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.720526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec503d3f-66c5-42fa-8ee1-1eb67cb7da22 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.720526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec503d3f-66c5-42fa-8ee1-1eb67cb7da22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.737311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ec503d3f-66c5-42fa-8ee1-1eb67cb7da22 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:36.738500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ec503d3f-66c5-42fa-8ee1-1eb67cb7da22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:36.746977 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1335/2660] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:36.751831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c795bb-ff2f-447d-b273-a485b7e1667f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:36.763842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c795bb-ff2f-447d-b273-a485b7e1667f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:36.767254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e543754-cb89-4b03-af11-491b60d9c033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:36.768133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e543754-cb89-4b03-af11-491b60d9c033 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.768582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e543754-cb89-4b03-af11-491b60d9c033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.769022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e543754-cb89-4b03-af11-491b60d9c033 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.770271 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e543754-cb89-4b03-af11-491b60d9c033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:49:36.770751 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e543754-cb89-4b03-af11-491b60d9c033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:36.772959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e543754-cb89-4b03-af11-491b60d9c033 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:36.787732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1326/2661] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:49:36.793293 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec503d3f-66c5-42fa-8ee1-1eb67cb7da22 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:36.804301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec503d3f-66c5-42fa-8ee1-1eb67cb7da22 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:36.810519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eb714-148e-47aa-b1e7-15860d77b32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:36.811163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eb714-148e-47aa-b1e7-15860d77b32c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.811445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eb714-148e-47aa-b1e7-15860d77b32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.811716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eb714-148e-47aa-b1e7-15860d77b32c admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.812530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357eb714-148e-47aa-b1e7-15860d77b32c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:36.814524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357eb714-148e-47aa-b1e7-15860d77b32c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:36.915261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1336/2662] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:36 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:36.920048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e543754-cb89-4b03-af11-491b60d9c033 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:36.930468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e543754-cb89-4b03-af11-491b60d9c033 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:36.993587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad22c3-230c-4676-b31d-1cddbba95488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:36.994192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad22c3-230c-4676-b31d-1cddbba95488 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:36.994472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad22c3-230c-4676-b31d-1cddbba95488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:36.994749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad22c3-230c-4676-b31d-1cddbba95488 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:36.995424 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad22c3-230c-4676-b31d-1cddbba95488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:36.997449 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad22c3-230c-4676-b31d-1cddbba95488 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:37.090842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1327/2663] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:36 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:37.095058 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eb714-148e-47aa-b1e7-15860d77b32c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:37.106292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eb714-148e-47aa-b1e7-15860d77b32c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:37.110887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47654100-e8c7-4749-8d44-98cb5e0fe148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:37.111494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654100-e8c7-4749-8d44-98cb5e0fe148 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:37.111777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654100-e8c7-4749-8d44-98cb5e0fe148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:37.112108 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654100-e8c7-4749-8d44-98cb5e0fe148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:37.112856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47654100-e8c7-4749-8d44-98cb5e0fe148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:37.114458 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47654100-e8c7-4749-8d44-98cb5e0fe148 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:37.125069 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1337/2664] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:37.129127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad22c3-230c-4676-b31d-1cddbba95488 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:37.139409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad22c3-230c-4676-b31d-1cddbba95488 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:37.148183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0af492-2544-435c-8b85-1531945b508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:37.148183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af492-2544-435c-8b85-1531945b508d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:37.148183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af492-2544-435c-8b85-1531945b508d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:37.148183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af492-2544-435c-8b85-1531945b508d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:37.148619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0af492-2544-435c-8b85-1531945b508d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:37.148619 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0af492-2544-435c-8b85-1531945b508d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:37.159587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1328/2665] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:37.164472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47654100-e8c7-4749-8d44-98cb5e0fe148 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:37.177753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47654100-e8c7-4749-8d44-98cb5e0fe148 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:37.182554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f736a-9fa0-4d92-af29-058aa0e7f104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:37.183224 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f736a-9fa0-4d92-af29-058aa0e7f104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:37.183510 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f736a-9fa0-4d92-af29-058aa0e7f104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:37.183787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f736a-9fa0-4d92-af29-058aa0e7f104 admin admin] PATH_INFO: `/v3/projects/2bfad345b6c74cdab42277563e493080/users/38ceb77271324bd79d587471bb15b7e0/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:37.199100 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f736a-9fa0-4d92-af29-058aa0e7f104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bfad345b6c74cdab42277563e493080, user_id=38ceb77271324bd79d587471bb15b7e0, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:37.201841 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f736a-9fa0-4d92-af29-058aa0e7f104 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:37.302414 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1338/2666] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:37 2022] PUT /identity/v3/projects/2bfad345b6c74cdab42277563e493080/users/38ceb77271324bd79d587471bb15b7e0/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:37.317582 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92ae7f-5722-42df-882f-9aa88e788b09 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:37.317868 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92ae7f-5722-42df-882f-9aa88e788b09 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:37.318143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92ae7f-5722-42df-882f-9aa88e788b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:37.345804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1c92ae7f-5722-42df-882f-9aa88e788b09 None None] MFA Rules not processed for user `38ceb77271324bd79d587471bb15b7e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:37.346995 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1c92ae7f-5722-42df-882f-9aa88e788b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:37.377061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:37.431743 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1329/2667] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:37.432323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f736a-9fa0-4d92-af29-058aa0e7f104 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:37.442425 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:37.553361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f736a-9fa0-4d92-af29-058aa0e7f104 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:37.564823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f83b8-0dc8-40d4-aa03-abb2db9b2244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:37.565463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f83b8-0dc8-40d4-aa03-abb2db9b2244 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:37.565744 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f83b8-0dc8-40d4-aa03-abb2db9b2244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:37.566121 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f83b8-0dc8-40d4-aa03-abb2db9b2244 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:37.566836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f83b8-0dc8-40d4-aa03-abb2db9b2244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:37.568995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f83b8-0dc8-40d4-aa03-abb2db9b2244 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:37.658716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1339/2668] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:37 2022] POST /identity/v3/projects => generated 387 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:37.662600 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92ae7f-5722-42df-882f-9aa88e788b09 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:37.673559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92ae7f-5722-42df-882f-9aa88e788b09 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:37.730748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008cb06-7abb-441d-b956-2969e19351b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:37.731417 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008cb06-7abb-441d-b956-2969e19351b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:37.731621 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008cb06-7abb-441d-b956-2969e19351b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:37.731895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008cb06-7abb-441d-b956-2969e19351b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:37.732580 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008cb06-7abb-441d-b956-2969e19351b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:37.734490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008cb06-7abb-441d-b956-2969e19351b1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:37.787490 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1330/2669] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:37 2022] POST /identity/v3/users => generated 300 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:37.792046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f83b8-0dc8-40d4-aa03-abb2db9b2244 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:37.803103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f83b8-0dc8-40d4-aa03-abb2db9b2244 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:37.808564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5de3c9-ccf3-4e2d-8fb1-7ed2d0dec1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:37.809196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de3c9-ccf3-4e2d-8fb1-7ed2d0dec1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:37.809477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de3c9-ccf3-4e2d-8fb1-7ed2d0dec1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:37.809751 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de3c9-ccf3-4e2d-8fb1-7ed2d0dec1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:37.810432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5de3c9-ccf3-4e2d-8fb1-7ed2d0dec1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:37.812086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5de3c9-ccf3-4e2d-8fb1-7ed2d0dec1f4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:37.822810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1340/2670] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:37.827150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008cb06-7abb-441d-b956-2969e19351b1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:37.837488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008cb06-7abb-441d-b956-2969e19351b1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:37.842799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c51ad-6dd9-4966-a453-bc6828d0e99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:37.843460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c51ad-6dd9-4966-a453-bc6828d0e99b admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:37.843749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c51ad-6dd9-4966-a453-bc6828d0e99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:37.844056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c51ad-6dd9-4966-a453-bc6828d0e99b admin admin] PATH_INFO: `/v3/projects/d04966ee1e5a4be9b9228e369a614979/users/cb1fe90cc3244fd2907d380562127cc2/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:37.859085 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c51ad-6dd9-4966-a453-bc6828d0e99b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d04966ee1e5a4be9b9228e369a614979, user_id=cb1fe90cc3244fd2907d380562127cc2, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:37.861891 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c51ad-6dd9-4966-a453-bc6828d0e99b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:37.930381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1331/2671] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:37 2022] PUT /identity/v3/projects/d04966ee1e5a4be9b9228e369a614979/users/cb1fe90cc3244fd2907d380562127cc2/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:37.934605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5de3c9-ccf3-4e2d-8fb1-7ed2d0dec1f4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:37.945078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5de3c9-ccf3-4e2d-8fb1-7ed2d0dec1f4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:38.004885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c3f56-eda4-476a-875d-62766eca3bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:38.005544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c3f56-eda4-476a-875d-62766eca3bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:38.005824 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c3f56-eda4-476a-875d-62766eca3bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:38.006097 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c3f56-eda4-476a-875d-62766eca3bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:38.006783 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c3f56-eda4-476a-875d-62766eca3bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:38.008423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c3f56-eda4-476a-875d-62766eca3bc3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:38.018226 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1341/2672] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:38.023678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c51ad-6dd9-4966-a453-bc6828d0e99b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:38.033775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c51ad-6dd9-4966-a453-bc6828d0e99b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:38.038473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db227d-75af-4cf2-8ebb-a0d58fdd1aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:38.039065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db227d-75af-4cf2-8ebb-a0d58fdd1aea admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:38.039346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db227d-75af-4cf2-8ebb-a0d58fdd1aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:38.039618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db227d-75af-4cf2-8ebb-a0d58fdd1aea admin admin] PATH_INFO: `/v3/projects/d04966ee1e5a4be9b9228e369a614979/users/cb1fe90cc3244fd2907d380562127cc2/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:38.042412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db227d-75af-4cf2-8ebb-a0d58fdd1aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d04966ee1e5a4be9b9228e369a614979, user_id=cb1fe90cc3244fd2907d380562127cc2, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:38.045116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db227d-75af-4cf2-8ebb-a0d58fdd1aea admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:38.055484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1332/2673] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:38 2022] PUT /identity/v3/projects/d04966ee1e5a4be9b9228e369a614979/users/cb1fe90cc3244fd2907d380562127cc2/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:38.065868 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d4caa-0521-40c1-923d-374e6d3a6fa4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:38.066156 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d4caa-0521-40c1-923d-374e6d3a6fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:38.066442 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d4caa-0521-40c1-923d-374e6d3a6fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:38.094630 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d63d4caa-0521-40c1-923d-374e6d3a6fa4 None None] MFA Rules not processed for user `cb1fe90cc3244fd2907d380562127cc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:38.095843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d63d4caa-0521-40c1-923d-374e6d3a6fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:38.160015 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:38.163575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1342/2674] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:38.175223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db227d-75af-4cf2-8ebb-a0d58fdd1aea admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:38.177072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-83db227d-75af-4cf2-8ebb-a0d58fdd1aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:38.234595 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db227d-75af-4cf2-8ebb-a0d58fdd1aea admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:38.238247 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347294c7-9ff1-43bb-a76a-4e7e8c16b286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:38.238859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347294c7-9ff1-43bb-a76a-4e7e8c16b286 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:38.239147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347294c7-9ff1-43bb-a76a-4e7e8c16b286 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:38.239421 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347294c7-9ff1-43bb-a76a-4e7e8c16b286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:38.248456 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347294c7-9ff1-43bb-a76a-4e7e8c16b286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:38.249893 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347294c7-9ff1-43bb-a76a-4e7e8c16b286 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:38.259197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1333/2675] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:38.288835 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d4caa-0521-40c1-923d-374e6d3a6fa4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:38.351806 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d4caa-0521-40c1-923d-374e6d3a6fa4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:38.356659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456814f-46d3-4809-bb25-6657190b2a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:38.357338 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456814f-46d3-4809-bb25-6657190b2a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:38.357659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456814f-46d3-4809-bb25-6657190b2a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:38.357989 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456814f-46d3-4809-bb25-6657190b2a5d admin admin] PATH_INFO: `/v3/users/38ceb77271324bd79d587471bb15b7e0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:38.359206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456814f-46d3-4809-bb25-6657190b2a5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=38ceb77271324bd79d587471bb15b7e0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:38.361243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456814f-46d3-4809-bb25-6657190b2a5d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:38.569664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a456814f-46d3-4809-bb25-6657190b2a5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38ceb77271324bd79d587471bb15b7e0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:38.570421 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a456814f-46d3-4809-bb25-6657190b2a5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38ceb77271324bd79d587471bb15b7e0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:38.579893 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a456814f-46d3-4809-bb25-6657190b2a5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38ceb77271324bd79d587471bb15b7e0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:38.593207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a456814f-46d3-4809-bb25-6657190b2a5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38ceb77271324bd79d587471bb15b7e0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:38.629009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347294c7-9ff1-43bb-a76a-4e7e8c16b286 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:38.630627 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-347294c7-9ff1-43bb-a76a-4e7e8c16b286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:38.650231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347294c7-9ff1-43bb-a76a-4e7e8c16b286 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:38.654923 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:38.662624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:38.662624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:38.662624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] PATH_INFO: `/v3/users/4c648d4c6f254868bb56cc6bc8134ec7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:38.662624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c648d4c6f254868bb56cc6bc8134ec7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:38.662624 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:38.685619 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a456814f-46d3-4809-bb25-6657190b2a5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38ceb77271324bd79d587471bb15b7e0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:38.688919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:38.693383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1343/2676] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:38 2022] DELETE /identity/v3/users/38ceb77271324bd79d587471bb15b7e0 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:38.698733 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e488ff4-cb3d-4491-9379-80e78761d57a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:38.699205 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e488ff4-cb3d-4491-9379-80e78761d57a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:38.699482 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e488ff4-cb3d-4491-9379-80e78761d57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:38.719743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9e488ff4-cb3d-4491-9379-80e78761d57a None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:38.720990 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e488ff4-cb3d-4491-9379-80e78761d57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:38.784798 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1344/2677] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:38.795561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e488ff4-cb3d-4491-9379-80e78761d57a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:38.796998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e488ff4-cb3d-4491-9379-80e78761d57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:38.820395 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:38.857440 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e488ff4-cb3d-4491-9379-80e78761d57a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:38.861165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ca06b-529f-4adb-8d18-6fdb262ab43f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:38.861773 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ca06b-529f-4adb-8d18-6fdb262ab43f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:38.862053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ca06b-529f-4adb-8d18-6fdb262ab43f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:38.862328 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ca06b-529f-4adb-8d18-6fdb262ab43f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:38.872175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ca06b-529f-4adb-8d18-6fdb262ab43f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:38.873574 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ca06b-529f-4adb-8d18-6fdb262ab43f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:38.883180 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1345/2678] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:38.888625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c648d4c6f254868bb56cc6bc8134ec7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:38.889358 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c648d4c6f254868bb56cc6bc8134ec7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:38.889970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c648d4c6f254868bb56cc6bc8134ec7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:38.902655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c648d4c6f254868bb56cc6bc8134ec7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:38.982753 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c648d4c6f254868bb56cc6bc8134ec7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:38.996441 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1334/2679] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:38 2022] DELETE /identity/v3/users/4c648d4c6f254868bb56cc6bc8134ec7 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:39.000971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d644-a47a-43a0-9c55-862d3f5c35ca None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:39.001260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d644-a47a-43a0-9c55-862d3f5c35ca None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:39.001546 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d644-a47a-43a0-9c55-862d3f5c35ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:39.020456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9961d644-a47a-43a0-9c55-862d3f5c35ca None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:39.021652 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9961d644-a47a-43a0-9c55-862d3f5c35ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:39.085103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1346/2680] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:39.095744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:39.097311 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:39.187376 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c228b2-6cd1-4d79-9c8e-71773042756e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:39.192604 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21a855-f3bb-426c-85c0-3bfa427ea623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:39.193238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a855-f3bb-426c-85c0-3bfa427ea623 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:39.193601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a855-f3bb-426c-85c0-3bfa427ea623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:39.193884 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a855-f3bb-426c-85c0-3bfa427ea623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:39.204847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21a855-f3bb-426c-85c0-3bfa427ea623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:39.206511 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21a855-f3bb-426c-85c0-3bfa427ea623 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:39.222321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1335/2681] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:39.384932 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961d644-a47a-43a0-9c55-862d3f5c35ca None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:39.386505 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9961d644-a47a-43a0-9c55-862d3f5c35ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:39.398504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961d644-a47a-43a0-9c55-862d3f5c35ca None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:39.403847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:39.404531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:39.404817 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:39.405646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] PATH_INFO: `/v3/projects/2bfad345b6c74cdab42277563e493080` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:39.406687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bfad345b6c74cdab42277563e493080)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:39.408766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:39.531881 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bfad345b6c74cdab42277563e493080', 'request_id': 'req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:39.593201 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bfad345b6c74cdab42277563e493080', 'request_id': 'req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:39.603206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bfad345b6c74cdab42277563e493080', 'request_id': 'req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:39.620409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] The token cache is being invalidate because project 2bfad345b6c74cdab42277563e493080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:39.621024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:39.621857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:39.623789 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1347/2682] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:39 2022] DELETE /identity/v3/projects/2bfad345b6c74cdab42277563e493080 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:39.630764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21a855-f3bb-426c-85c0-3bfa427ea623 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:39.637532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21a855-f3bb-426c-85c0-3bfa427ea623 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:39.699109 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:39.699757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:39.700138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:39.700351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] PATH_INFO: `/v3/users/cb1fe90cc3244fd2907d380562127cc2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:39.701532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb1fe90cc3244fd2907d380562127cc2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:39.703444 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:39.830138 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:39.831611 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:39.856999 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748c9de-8a1a-460d-bd72-a8852e1b0b5a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:39.863890 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:39.864581 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:39.864861 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:39.865157 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] PATH_INFO: `/v3/projects/175e7a36d1e34c199eeb030b88cfccc5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:39.867334 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] RBAC: Authorizing `identity:delete_project(project_id=175e7a36d1e34c199eeb030b88cfccc5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:39.869320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:39.875970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:39.882780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb1fe90cc3244fd2907d380562127cc2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:39.883723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb1fe90cc3244fd2907d380562127cc2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:39.884617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb1fe90cc3244fd2907d380562127cc2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:39.899063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb1fe90cc3244fd2907d380562127cc2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.033571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb1fe90cc3244fd2907d380562127cc2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.045011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:40.046966 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1336/2683] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:39 2022] DELETE /identity/v3/users/cb1fe90cc3244fd2907d380562127cc2 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:40.052024 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175e7a36d1e34c199eeb030b88cfccc5', 'request_id': 'req-f926f629-7495-4add-bec9-8b84602b7aae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.153836 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175e7a36d1e34c199eeb030b88cfccc5', 'request_id': 'req-f926f629-7495-4add-bec9-8b84602b7aae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.159864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175e7a36d1e34c199eeb030b88cfccc5', 'request_id': 'req-f926f629-7495-4add-bec9-8b84602b7aae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.173006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] The token cache is being invalidate because project 175e7a36d1e34c199eeb030b88cfccc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:40.173297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.173942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.175432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1348/2684] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:39 2022] DELETE /identity/v3/projects/175e7a36d1e34c199eeb030b88cfccc5 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:40.179606 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:40.181183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:40.248393 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63489223-876a-4838-a7c1-01cd813ae8e1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:40.253290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:40.254086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:40.254485 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:40.254967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] PATH_INFO: `/v3/users/68356e32d7f74fad879111fb563d8350` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:40.256211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=68356e32d7f74fad879111fb563d8350)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:40.258122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:40.489121 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68356e32d7f74fad879111fb563d8350'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.489789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68356e32d7f74fad879111fb563d8350'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.490399 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68356e32d7f74fad879111fb563d8350'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.504257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68356e32d7f74fad879111fb563d8350'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.565530 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68356e32d7f74fad879111fb563d8350'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:40.580120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1337/2685] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:40 2022] DELETE /identity/v3/users/68356e32d7f74fad879111fb563d8350 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:40.703941 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:40.705441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:40.796551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926f629-7495-4add-bec9-8b84602b7aae admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:40.856394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:40.857354 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:40.857635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:40.857908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] PATH_INFO: `/v3/projects/d04966ee1e5a4be9b9228e369a614979` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:40.859800 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d04966ee1e5a4be9b9228e369a614979)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:40.861567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:40.980920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd04966ee1e5a4be9b9228e369a614979', 'request_id': 'req-5260daee-af2d-4c50-8ab6-57d29ad6b113'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.069855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:41.071467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:41.105887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd04966ee1e5a4be9b9228e369a614979', 'request_id': 'req-5260daee-af2d-4c50-8ab6-57d29ad6b113'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.119261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd04966ee1e5a4be9b9228e369a614979', 'request_id': 'req-5260daee-af2d-4c50-8ab6-57d29ad6b113'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.131211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] The token cache is being invalidate because project d04966ee1e5a4be9b9228e369a614979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:41.131502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.131667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:41.132184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5260daee-af2d-4c50-8ab6-57d29ad6b113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.133964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1349/2686] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:40 2022] DELETE /identity/v3/projects/d04966ee1e5a4be9b9228e369a614979 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:41.138891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9769446-4341-4b94-8431-39eb36177f42 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:41.140038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9769446-4341-4b94-8431-39eb36177f42 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:41.140450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9769446-4341-4b94-8431-39eb36177f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:41.154866 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:41.175919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa06bc0-9e5a-40da-8a48-e57819e2e2ce admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:41.224374 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d9769446-4341-4b94-8431-39eb36177f42 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:41.225971 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d9769446-4341-4b94-8431-39eb36177f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:41.240815 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:41.241549 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:41.241901 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:41.242344 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] PATH_INFO: `/v3/projects/f6142a3500bb4763999a7dc2d764ab91` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:41.243587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6142a3500bb4763999a7dc2d764ab91)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:41.245861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:41.284899 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1350/2687] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:41.293524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f098ec-d655-4602-9aef-f68e604b7ace None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:41.293749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f098ec-d655-4602-9aef-f68e604b7ace None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:41.294027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f098ec-d655-4602-9aef-f68e604b7ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:41.312832 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-22f098ec-d655-4602-9aef-f68e604b7ace None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:41.316869 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-22f098ec-d655-4602-9aef-f68e604b7ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:41.322219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1351/2688] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:41.326923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f098ec-d655-4602-9aef-f68e604b7ace None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:41.335939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f098ec-d655-4602-9aef-f68e604b7ace None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:41.341518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de450a2d-657b-43f7-86d4-e73abfd70259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:41.342104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de450a2d-657b-43f7-86d4-e73abfd70259 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:41.342387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de450a2d-657b-43f7-86d4-e73abfd70259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:41.342664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de450a2d-657b-43f7-86d4-e73abfd70259 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:41.343350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de450a2d-657b-43f7-86d4-e73abfd70259 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:49:41.343679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de450a2d-657b-43f7-86d4-e73abfd70259 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:41.345321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de450a2d-657b-43f7-86d4-e73abfd70259 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:41.358365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1352/2689] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:49:41.362289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de450a2d-657b-43f7-86d4-e73abfd70259 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:41.371411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6142a3500bb4763999a7dc2d764ab91', 'request_id': 'req-3df06e11-9461-4a0a-9b67-a1b685557e18'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.376234 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de450a2d-657b-43f7-86d4-e73abfd70259 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:41.379809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6142a3500bb4763999a7dc2d764ab91', 'request_id': 'req-3df06e11-9461-4a0a-9b67-a1b685557e18'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.434797 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f7795-1184-41ef-b96c-6e6d02c93ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:41.435419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f7795-1184-41ef-b96c-6e6d02c93ffe admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:41.435700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f7795-1184-41ef-b96c-6e6d02c93ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:41.436002 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f7795-1184-41ef-b96c-6e6d02c93ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:41.436717 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f7795-1184-41ef-b96c-6e6d02c93ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:41.438743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f7795-1184-41ef-b96c-6e6d02c93ffe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:41.456788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6142a3500bb4763999a7dc2d764ab91', 'request_id': 'req-3df06e11-9461-4a0a-9b67-a1b685557e18'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.470262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] The token cache is being invalidate because project f6142a3500bb4763999a7dc2d764ab91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:41.470550 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.471205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.472778 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1338/2690] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:41 2022] DELETE /identity/v3/projects/f6142a3500bb4763999a7dc2d764ab91 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:41.476613 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:41.477910 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:41.490671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df06e11-9461-4a0a-9b67-a1b685557e18 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:41.495560 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434cc98-54c7-46b2-aff1-983bb3c024ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:41.496253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434cc98-54c7-46b2-aff1-983bb3c024ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:41.496542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434cc98-54c7-46b2-aff1-983bb3c024ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:41.496825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434cc98-54c7-46b2-aff1-983bb3c024ea admin admin] PATH_INFO: `/v3/users/e56c688c8c9a4067acbd030137131bc6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:41.498038 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434cc98-54c7-46b2-aff1-983bb3c024ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=e56c688c8c9a4067acbd030137131bc6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:41.500047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434cc98-54c7-46b2-aff1-983bb3c024ea admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:41.532746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1353/2691] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:41 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:41.536903 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f7795-1184-41ef-b96c-6e6d02c93ffe admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:41.538552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f99f7795-1184-41ef-b96c-6e6d02c93ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:41.601539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f7795-1184-41ef-b96c-6e6d02c93ffe admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:41.606424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29662c-617c-4e59-98ab-e531544a3a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:41.607056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29662c-617c-4e59-98ab-e531544a3a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:41.613006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29662c-617c-4e59-98ab-e531544a3a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:41.613006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29662c-617c-4e59-98ab-e531544a3a31 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:41.613006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29662c-617c-4e59-98ab-e531544a3a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:41.613006 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29662c-617c-4e59-98ab-e531544a3a31 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:41.725870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b434cc98-54c7-46b2-aff1-983bb3c024ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e56c688c8c9a4067acbd030137131bc6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.726476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:41.726953 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b434cc98-54c7-46b2-aff1-983bb3c024ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e56c688c8c9a4067acbd030137131bc6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.727218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b434cc98-54c7-46b2-aff1-983bb3c024ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e56c688c8c9a4067acbd030137131bc6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.732982 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1354/2692] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:41 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:41.737019 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29662c-617c-4e59-98ab-e531544a3a31 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:41.738178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:41.738826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ac29662c-617c-4e59-98ab-e531544a3a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:41.740516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b434cc98-54c7-46b2-aff1-983bb3c024ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e56c688c8c9a4067acbd030137131bc6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.751107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29662c-617c-4e59-98ab-e531544a3a31 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:41.755776 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5bb31e-c04c-47eb-80e7-6e0c1d4445cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:41.756412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bb31e-c04c-47eb-80e7-6e0c1d4445cf admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:41.756695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bb31e-c04c-47eb-80e7-6e0c1d4445cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:41.756972 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bb31e-c04c-47eb-80e7-6e0c1d4445cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:41.757660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bb31e-c04c-47eb-80e7-6e0c1d4445cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:41.759249 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bb31e-c04c-47eb-80e7-6e0c1d4445cf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:41.769449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1355/2693] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:41.774761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5bb31e-c04c-47eb-80e7-6e0c1d4445cf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:41.783153 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5bb31e-c04c-47eb-80e7-6e0c1d4445cf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:41.787844 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387497b2-b865-4922-9bfe-9a6f71267cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:41.788453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387497b2-b865-4922-9bfe-9a6f71267cab admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:41.788738 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387497b2-b865-4922-9bfe-9a6f71267cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:41.789014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387497b2-b865-4922-9bfe-9a6f71267cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:41.789686 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387497b2-b865-4922-9bfe-9a6f71267cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:41.791365 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387497b2-b865-4922-9bfe-9a6f71267cab admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:41.801612 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1356/2694] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:49:41.806038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387497b2-b865-4922-9bfe-9a6f71267cab admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:41.816062 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387497b2-b865-4922-9bfe-9a6f71267cab admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:41.820689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8280e-cdfe-4359-9f3a-846e59f30137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:41.821299 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8280e-cdfe-4359-9f3a-846e59f30137 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:41.821584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8280e-cdfe-4359-9f3a-846e59f30137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:41.821862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8280e-cdfe-4359-9f3a-846e59f30137 admin admin] PATH_INFO: `/v3/projects/a4ea90bd95e64e86b077f3717f9fe8d0/users/85c9d310c44b4b889389809bccce0cd2/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:41.827533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b434cc98-54c7-46b2-aff1-983bb3c024ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e56c688c8c9a4067acbd030137131bc6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:41.838689 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8280e-cdfe-4359-9f3a-846e59f30137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ea90bd95e64e86b077f3717f9fe8d0, user_id=85c9d310c44b4b889389809bccce0cd2, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:41.840969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1339/2695] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:41 2022] DELETE /identity/v3/users/e56c688c8c9a4067acbd030137131bc6 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:41.841630 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8280e-cdfe-4359-9f3a-846e59f30137 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:41.931195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1357/2696] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:41 2022] PUT /identity/v3/projects/a4ea90bd95e64e86b077f3717f9fe8d0/users/85c9d310c44b4b889389809bccce0cd2/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:41.942575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08838-3a51-48d0-9835-a947d0ae5312 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:41.942912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08838-3a51-48d0-9835-a947d0ae5312 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:41.943214 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08838-3a51-48d0-9835-a947d0ae5312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:41.973138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9ef08838-3a51-48d0-9835-a947d0ae5312 None None] MFA Rules not processed for user `85c9d310c44b4b889389809bccce0cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:41.974377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9ef08838-3a51-48d0-9835-a947d0ae5312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:42.038001 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1340/2697] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:42.050086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8280e-cdfe-4359-9f3a-846e59f30137 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:42.051792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e4d8280e-cdfe-4359-9f3a-846e59f30137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:42.135615 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8280e-cdfe-4359-9f3a-846e59f30137 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:42.139364 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e97ac9-55ee-44fc-b10c-38f96fa8f4ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:42.140030 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e97ac9-55ee-44fc-b10c-38f96fa8f4ae service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:42.140313 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e97ac9-55ee-44fc-b10c-38f96fa8f4ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:42.140587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e97ac9-55ee-44fc-b10c-38f96fa8f4ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:42.149987 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e97ac9-55ee-44fc-b10c-38f96fa8f4ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:42.151942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e97ac9-55ee-44fc-b10c-38f96fa8f4ae service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:42.161542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1358/2698] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:49:42.181922 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef08838-3a51-48d0-9835-a947d0ae5312 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:42.252691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef08838-3a51-48d0-9835-a947d0ae5312 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:42.257237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea272c-0d15-4233-b25b-78c2446ae0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:42.257878 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea272c-0d15-4233-b25b-78c2446ae0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:42.258158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea272c-0d15-4233-b25b-78c2446ae0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:42.258433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea272c-0d15-4233-b25b-78c2446ae0f3 admin admin] PATH_INFO: `/v3/users/85c9d310c44b4b889389809bccce0cd2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:42.259773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea272c-0d15-4233-b25b-78c2446ae0f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85c9d310c44b4b889389809bccce0cd2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:42.261823 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea272c-0d15-4233-b25b-78c2446ae0f3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:42.317663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e97ac9-55ee-44fc-b10c-38f96fa8f4ae service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:42.319081 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-31e97ac9-55ee-44fc-b10c-38f96fa8f4ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:42.333494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e97ac9-55ee-44fc-b10c-38f96fa8f4ae service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:42.339303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:42.339954 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:42.340262 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:42.340534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] PATH_INFO: `/v3/projects/2d90076372974667a1cc5499a472411c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:42.341854 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d90076372974667a1cc5499a472411c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:42.343745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:42.454196 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-75ea272c-0d15-4233-b25b-78c2446ae0f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85c9d310c44b4b889389809bccce0cd2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:42.454896 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-75ea272c-0d15-4233-b25b-78c2446ae0f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85c9d310c44b4b889389809bccce0cd2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:42.455554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-75ea272c-0d15-4233-b25b-78c2446ae0f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85c9d310c44b4b889389809bccce0cd2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:42.470894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-75ea272c-0d15-4233-b25b-78c2446ae0f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85c9d310c44b4b889389809bccce0cd2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:42.476297 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:42.489139 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d90076372974667a1cc5499a472411c', 'request_id': 'req-7b7aeba1-4354-4f2e-a058-738db4a0ce58'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:42.582389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-75ea272c-0d15-4233-b25b-78c2446ae0f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85c9d310c44b4b889389809bccce0cd2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:42.595865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1341/2699] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:42 2022] DELETE /identity/v3/users/85c9d310c44b4b889389809bccce0cd2 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:42.600554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aaf6a-0128-4401-8cb1-b05decb2632c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:42.600844 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aaf6a-0128-4401-8cb1-b05decb2632c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:42.601123 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aaf6a-0128-4401-8cb1-b05decb2632c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:42.623355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-058aaf6a-0128-4401-8cb1-b05decb2632c None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:42.624601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-058aaf6a-0128-4401-8cb1-b05decb2632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:42.687271 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1342/2700] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:42.696946 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aaf6a-0128-4401-8cb1-b05decb2632c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:42.698488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-058aaf6a-0128-4401-8cb1-b05decb2632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:42.714544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d90076372974667a1cc5499a472411c', 'request_id': 'req-7b7aeba1-4354-4f2e-a058-738db4a0ce58'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:42.721000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d90076372974667a1cc5499a472411c', 'request_id': 'req-7b7aeba1-4354-4f2e-a058-738db4a0ce58'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:42.722841 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:42.733930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] The token cache is being invalidate because project 2d90076372974667a1cc5499a472411c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:42.734542 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:42.735247 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7b7aeba1-4354-4f2e-a058-738db4a0ce58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:42.740764 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1359/2701] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:42 2022] DELETE /identity/v3/projects/2d90076372974667a1cc5499a472411c => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:42.742053 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2de7b-7e28-4c34-ad4c-edb98cd50a2e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:42.742409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2de7b-7e28-4c34-ad4c-edb98cd50a2e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:42.742775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2de7b-7e28-4c34-ad4c-edb98cd50a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:42.758629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aaf6a-0128-4401-8cb1-b05decb2632c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:42.762389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e2b2de7b-7e28-4c34-ad4c-edb98cd50a2e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:42.763108 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56034842-fb4a-4a09-87ce-7f9467f8487e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:42.763782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56034842-fb4a-4a09-87ce-7f9467f8487e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:42.763913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e2b2de7b-7e28-4c34-ad4c-edb98cd50a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:42.764120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56034842-fb4a-4a09-87ce-7f9467f8487e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:42.775086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56034842-fb4a-4a09-87ce-7f9467f8487e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:42.775086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-56034842-fb4a-4a09-87ce-7f9467f8487e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:42.775298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1360/2702] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:42.780354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56034842-fb4a-4a09-87ce-7f9467f8487e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:42.782151 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56034842-fb4a-4a09-87ce-7f9467f8487e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:42.782292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1da03-8f50-485f-aa96-e540a5e3ebf3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:42.782474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1da03-8f50-485f-aa96-e540a5e3ebf3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:42.782761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1da03-8f50-485f-aa96-e540a5e3ebf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:42.792436 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1343/2703] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:49:42.802275 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-09b1da03-8f50-485f-aa96-e540a5e3ebf3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:42.803622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-09b1da03-8f50-485f-aa96-e540a5e3ebf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:42.812314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1361/2704] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:42.816575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56034842-fb4a-4a09-87ce-7f9467f8487e service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:42.827161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56034842-fb4a-4a09-87ce-7f9467f8487e service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:42.832400 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d383ed1-e204-4657-a511-3ac612af325e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:42.832881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d383ed1-e204-4657-a511-3ac612af325e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:42.833194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d383ed1-e204-4657-a511-3ac612af325e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:42.833767 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d383ed1-e204-4657-a511-3ac612af325e admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:42.834195 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d383ed1-e204-4657-a511-3ac612af325e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:49:42.834497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d383ed1-e204-4657-a511-3ac612af325e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:42.836099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d383ed1-e204-4657-a511-3ac612af325e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:42.848618 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1344/2705] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:49:42.853158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1da03-8f50-485f-aa96-e540a5e3ebf3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:42.862519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1da03-8f50-485f-aa96-e540a5e3ebf3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:42.867726 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b2f99-f564-48a6-ab7f-e6cf1f1674de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:42.868358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b2f99-f564-48a6-ab7f-e6cf1f1674de admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:42.868638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b2f99-f564-48a6-ab7f-e6cf1f1674de admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:42.868908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b2f99-f564-48a6-ab7f-e6cf1f1674de admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:42.869613 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b2f99-f564-48a6-ab7f-e6cf1f1674de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:42.871508 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b2f99-f564-48a6-ab7f-e6cf1f1674de admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:42.999567 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1362/2706] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:42 2022] POST /identity/v3/projects => generated 391 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:43.000810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d383ed1-e204-4657-a511-3ac612af325e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:43.011521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d383ed1-e204-4657-a511-3ac612af325e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:43.076629 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265173f-d99f-42f4-a941-ede1f4c20ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:43.077119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265173f-d99f-42f4-a941-ede1f4c20ddd admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.077414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265173f-d99f-42f4-a941-ede1f4c20ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.077691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265173f-d99f-42f4-a941-ede1f4c20ddd admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.078390 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265173f-d99f-42f4-a941-ede1f4c20ddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:43.080611 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265173f-d99f-42f4-a941-ede1f4c20ddd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:43.148999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1345/2707] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:42 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:43.153233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b2f99-f564-48a6-ab7f-e6cf1f1674de admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:43.163874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b2f99-f564-48a6-ab7f-e6cf1f1674de admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:43.169074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed0252-cedb-498c-b0d3-b96e67f0e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:43.169810 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed0252-cedb-498c-b0d3-b96e67f0e9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.170099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed0252-cedb-498c-b0d3-b96e67f0e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.170380 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed0252-cedb-498c-b0d3-b96e67f0e9d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.171066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed0252-cedb-498c-b0d3-b96e67f0e9d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:43.172708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed0252-cedb-498c-b0d3-b96e67f0e9d2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:43.183344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1363/2708] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:49:43.187350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265173f-d99f-42f4-a941-ede1f4c20ddd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:43.197201 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265173f-d99f-42f4-a941-ede1f4c20ddd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:43.202355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d1c84-0d61-4115-9991-d14d55325eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:43.202964 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d1c84-0d61-4115-9991-d14d55325eff admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.203245 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d1c84-0d61-4115-9991-d14d55325eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.203523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d1c84-0d61-4115-9991-d14d55325eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.204550 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d1c84-0d61-4115-9991-d14d55325eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:43.206183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d1c84-0d61-4115-9991-d14d55325eff admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:43.221495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1346/2709] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:43.221495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:43.221806 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed0252-cedb-498c-b0d3-b96e67f0e9d2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:43.230506 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed0252-cedb-498c-b0d3-b96e67f0e9d2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:43.234978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec98a49-3f10-4cfc-9c10-3c6c1cf30b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:43.235627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec98a49-3f10-4cfc-9c10-3c6c1cf30b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.235910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec98a49-3f10-4cfc-9c10-3c6c1cf30b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.236219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec98a49-3f10-4cfc-9c10-3c6c1cf30b2c admin admin] PATH_INFO: `/v3/projects/579a54ee385143bfbd37515edd7b8a09/users/da8e9a3148cd4a62988f9cb332289c54/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.252563 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec98a49-3f10-4cfc-9c10-3c6c1cf30b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=579a54ee385143bfbd37515edd7b8a09, user_id=da8e9a3148cd4a62988f9cb332289c54, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:43.255288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec98a49-3f10-4cfc-9c10-3c6c1cf30b2c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:43.264912 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d1c84-0d61-4115-9991-d14d55325eff admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:43.266638 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-971d1c84-0d61-4115-9991-d14d55325eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:43.279516 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d1c84-0d61-4115-9991-d14d55325eff admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:43.284231 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:43.284877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.290572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.290572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] PATH_INFO: `/v3/projects/a4ea90bd95e64e86b077f3717f9fe8d0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.290572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4ea90bd95e64e86b077f3717f9fe8d0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:43.290572 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:43.313673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1364/2710] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:43 2022] PUT /identity/v3/projects/579a54ee385143bfbd37515edd7b8a09/users/da8e9a3148cd4a62988f9cb332289c54/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:43.324248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eac496-dffc-4616-83ec-be9389d659f5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.324533 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eac496-dffc-4616-83ec-be9389d659f5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.324826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eac496-dffc-4616-83ec-be9389d659f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.351773 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-32eac496-dffc-4616-83ec-be9389d659f5 None None] MFA Rules not processed for user `da8e9a3148cd4a62988f9cb332289c54`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:43.353059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-32eac496-dffc-4616-83ec-be9389d659f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:43.391055 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4ea90bd95e64e86b077f3717f9fe8d0', 'request_id': 'req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:43.397518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4ea90bd95e64e86b077f3717f9fe8d0', 'request_id': 'req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:43.418349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1365/2711] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:43.428145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eac496-dffc-4616-83ec-be9389d659f5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:43.429509 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-32eac496-dffc-4616-83ec-be9389d659f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:43.456939 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4ea90bd95e64e86b077f3717f9fe8d0', 'request_id': 'req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:43.471065 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] The token cache is being invalidate because project a4ea90bd95e64e86b077f3717f9fe8d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:43.471359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:43.476244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ee82c95-8cf1-47a1-9bfd-83c849d731ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:43.476244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1347/2712] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:43 2022] DELETE /identity/v3/projects/a4ea90bd95e64e86b077f3717f9fe8d0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:43.479569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dfe56-ebee-42c3-9a8d-58e724b91776 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.480708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dfe56-ebee-42c3-9a8d-58e724b91776 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.480708 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dfe56-ebee-42c3-9a8d-58e724b91776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.497586 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eac496-dffc-4616-83ec-be9389d659f5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:43.502685 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4d9dfe56-ebee-42c3-9a8d-58e724b91776 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:43.504982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4d9dfe56-ebee-42c3-9a8d-58e724b91776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:43.534564 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:43.559061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25137-cf26-4bcf-adcb-73b3a566b515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:43.559691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25137-cf26-4bcf-adcb-73b3a566b515 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.559975 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25137-cf26-4bcf-adcb-73b3a566b515 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.560279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25137-cf26-4bcf-adcb-73b3a566b515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.562081 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5cd25137-cf26-4bcf-adcb-73b3a566b515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:43.582829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1348/2713] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:43.591577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8595066-9c6b-4c20-8396-3b631d8c8fc7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.591961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8595066-9c6b-4c20-8396-3b631d8c8fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.592338 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8595066-9c6b-4c20-8396-3b631d8c8fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.614176 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d8595066-9c6b-4c20-8396-3b631d8c8fc7 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:43.620328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d8595066-9c6b-4c20-8396-3b631d8c8fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:43.625244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1349/2714] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:43.629265 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8595066-9c6b-4c20-8396-3b631d8c8fc7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:43.634116 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25137-cf26-4bcf-adcb-73b3a566b515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:43.635692 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25137-cf26-4bcf-adcb-73b3a566b515 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:43.640294 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8595066-9c6b-4c20-8396-3b631d8c8fc7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:43.645473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418ada5-a129-47b7-a9a1-cba1e2f456b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:43.646144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418ada5-a129-47b7-a9a1-cba1e2f456b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.646429 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418ada5-a129-47b7-a9a1-cba1e2f456b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.646828 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1366/2715] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:49:43.646943 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418ada5-a129-47b7-a9a1-cba1e2f456b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.647395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418ada5-a129-47b7-a9a1-cba1e2f456b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:49:43.647702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418ada5-a129-47b7-a9a1-cba1e2f456b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:43.649446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418ada5-a129-47b7-a9a1-cba1e2f456b8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:43.661357 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1350/2716] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:49:43.670312 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25137-cf26-4bcf-adcb-73b3a566b515 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:43.686463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25137-cf26-4bcf-adcb-73b3a566b515 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:43.695768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41281baa-6a2b-4e09-8491-d064885ae5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:43.695768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41281baa-6a2b-4e09-8491-d064885ae5cd admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.695768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41281baa-6a2b-4e09-8491-d064885ae5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.695768 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41281baa-6a2b-4e09-8491-d064885ae5cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.696188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41281baa-6a2b-4e09-8491-d064885ae5cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:43.696464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41281baa-6a2b-4e09-8491-d064885ae5cd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:43.795900 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1367/2717] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:43 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:43.800200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418ada5-a129-47b7-a9a1-cba1e2f456b8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:43.810761 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418ada5-a129-47b7-a9a1-cba1e2f456b8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:43.869127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade74510-ed65-41b3-955f-70119d4b5492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:43.869889 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade74510-ed65-41b3-955f-70119d4b5492 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:43.870211 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade74510-ed65-41b3-955f-70119d4b5492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:43.870512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade74510-ed65-41b3-955f-70119d4b5492 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:43.871178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade74510-ed65-41b3-955f-70119d4b5492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:43.873175 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade74510-ed65-41b3-955f-70119d4b5492 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:43.881319 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:44.029359 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1351/2718] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:43 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:44.033480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41281baa-6a2b-4e09-8491-d064885ae5cd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:44.044698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41281baa-6a2b-4e09-8491-d064885ae5cd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:44.049605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92df211-bd0c-45e0-be61-eef7b6db4d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:44.050314 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df211-bd0c-45e0-be61-eef7b6db4d9f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:44.050634 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df211-bd0c-45e0-be61-eef7b6db4d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:44.050933 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df211-bd0c-45e0-be61-eef7b6db4d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:44.051625 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92df211-bd0c-45e0-be61-eef7b6db4d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:44.053394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92df211-bd0c-45e0-be61-eef7b6db4d9f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:44.065360 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1368/2719] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:44.069456 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade74510-ed65-41b3-955f-70119d4b5492 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:44.080419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade74510-ed65-41b3-955f-70119d4b5492 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:44.085256 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76528b-30d4-4e1a-888c-da88c8edcfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:44.085871 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76528b-30d4-4e1a-888c-da88c8edcfdc admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:44.091717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76528b-30d4-4e1a-888c-da88c8edcfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:44.091717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76528b-30d4-4e1a-888c-da88c8edcfdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:44.091717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76528b-30d4-4e1a-888c-da88c8edcfdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:44.091717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76528b-30d4-4e1a-888c-da88c8edcfdc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:44.101179 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1352/2720] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:44.106121 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92df211-bd0c-45e0-be61-eef7b6db4d9f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:44.116519 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92df211-bd0c-45e0-be61-eef7b6db4d9f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:44.121728 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12614c8-fe4d-4bff-956a-fd6e2b39b75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:44.122405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12614c8-fe4d-4bff-956a-fd6e2b39b75d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:44.122695 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12614c8-fe4d-4bff-956a-fd6e2b39b75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:44.122968 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12614c8-fe4d-4bff-956a-fd6e2b39b75d admin admin] PATH_INFO: `/v3/projects/96ed29d4572a4e47a53cab22abce9f47/users/e931214042b542ceb969ffea0cf41560/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:44.138600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12614c8-fe4d-4bff-956a-fd6e2b39b75d admin admin] RBAC: Authorizing `identity:create_grant(project_id=96ed29d4572a4e47a53cab22abce9f47, user_id=e931214042b542ceb969ffea0cf41560, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:44.141297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12614c8-fe4d-4bff-956a-fd6e2b39b75d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:44.217435 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1369/2721] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:44 2022] PUT /identity/v3/projects/96ed29d4572a4e47a53cab22abce9f47/users/e931214042b542ceb969ffea0cf41560/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:44.229259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5431a-a6cc-4acd-aff0-53b2ae998d1d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:44.230069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5431a-a6cc-4acd-aff0-53b2ae998d1d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:44.230069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5431a-a6cc-4acd-aff0-53b2ae998d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:44.257236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-15f5431a-a6cc-4acd-aff0-53b2ae998d1d None None] MFA Rules not processed for user `e931214042b542ceb969ffea0cf41560`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:44.258584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-15f5431a-a6cc-4acd-aff0-53b2ae998d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:44.340217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1353/2722] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:44.341321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12614c8-fe4d-4bff-956a-fd6e2b39b75d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:44.406627 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12614c8-fe4d-4bff-956a-fd6e2b39b75d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:44.411620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44fee7-4747-40f4-9d57-b6d6c734882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:44.412017 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44fee7-4747-40f4-9d57-b6d6c734882d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:44.412326 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44fee7-4747-40f4-9d57-b6d6c734882d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:44.412574 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44fee7-4747-40f4-9d57-b6d6c734882d admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:44.413255 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44fee7-4747-40f4-9d57-b6d6c734882d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:44.415188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44fee7-4747-40f4-9d57-b6d6c734882d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:44.492259 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1370/2723] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:44 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:44.496959 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5431a-a6cc-4acd-aff0-53b2ae998d1d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:44.507609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5431a-a6cc-4acd-aff0-53b2ae998d1d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:44.567970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fd515-9b8b-4e57-90e1-bc6fce6eb4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:44.568617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fd515-9b8b-4e57-90e1-bc6fce6eb4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:44.568902 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fd515-9b8b-4e57-90e1-bc6fce6eb4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:44.569181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fd515-9b8b-4e57-90e1-bc6fce6eb4b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:44.569856 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fd515-9b8b-4e57-90e1-bc6fce6eb4b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:44.571770 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fd515-9b8b-4e57-90e1-bc6fce6eb4b8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:44.680012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1354/2724] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:44 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:44.684839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44fee7-4747-40f4-9d57-b6d6c734882d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:44.695495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44fee7-4747-40f4-9d57-b6d6c734882d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:44.701691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0c52b-4467-4049-83ee-f1efe5ca94cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:44.702316 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0c52b-4467-4049-83ee-f1efe5ca94cc admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:44.702600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0c52b-4467-4049-83ee-f1efe5ca94cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:44.702876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0c52b-4467-4049-83ee-f1efe5ca94cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:44.703575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0c52b-4467-4049-83ee-f1efe5ca94cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:44.705709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0c52b-4467-4049-83ee-f1efe5ca94cc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:44.717746 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1371/2725] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:44.722576 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fd515-9b8b-4e57-90e1-bc6fce6eb4b8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:44.733061 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fd515-9b8b-4e57-90e1-bc6fce6eb4b8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:44.737945 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28193e97-c7cc-4641-9987-0a45d19d597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:44.738569 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28193e97-c7cc-4641-9987-0a45d19d597a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:44.738888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28193e97-c7cc-4641-9987-0a45d19d597a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:44.739135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28193e97-c7cc-4641-9987-0a45d19d597a admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:44.739864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28193e97-c7cc-4641-9987-0a45d19d597a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:44.741511 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28193e97-c7cc-4641-9987-0a45d19d597a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:44.752669 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1355/2726] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:44.756894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0c52b-4467-4049-83ee-f1efe5ca94cc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:44.762729 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:44.768376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0c52b-4467-4049-83ee-f1efe5ca94cc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:44.773212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3202b0-f1c7-4bd2-baee-a11e3466faf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:44.777909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3202b0-f1c7-4bd2-baee-a11e3466faf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:44.777909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3202b0-f1c7-4bd2-baee-a11e3466faf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:44.777909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3202b0-f1c7-4bd2-baee-a11e3466faf4 admin admin] PATH_INFO: `/v3/projects/bc193a04208145ae8d5fcdc843435808/users/66da15ccc1cd451c91629107678ef3d1/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:44.789520 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3202b0-f1c7-4bd2-baee-a11e3466faf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc193a04208145ae8d5fcdc843435808, user_id=66da15ccc1cd451c91629107678ef3d1, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:44.792350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3202b0-f1c7-4bd2-baee-a11e3466faf4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:44.908698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1372/2727] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:44 2022] PUT /identity/v3/projects/bc193a04208145ae8d5fcdc843435808/users/66da15ccc1cd451c91629107678ef3d1/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:44.919016 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809caf36-eeeb-4dfb-b66d-3c2af61cca7d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:44.919310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809caf36-eeeb-4dfb-b66d-3c2af61cca7d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:44.919592 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809caf36-eeeb-4dfb-b66d-3c2af61cca7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:44.947308 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-809caf36-eeeb-4dfb-b66d-3c2af61cca7d None None] MFA Rules not processed for user `66da15ccc1cd451c91629107678ef3d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:44.949238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-809caf36-eeeb-4dfb-b66d-3c2af61cca7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:44.983133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:45.019238 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1356/2728] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:45.020382 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3202b0-f1c7-4bd2-baee-a11e3466faf4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:45.092402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3202b0-f1c7-4bd2-baee-a11e3466faf4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:45.100125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60a901-d7f5-473d-ae0e-d5d570ec8942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:45.100125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60a901-d7f5-473d-ae0e-d5d570ec8942 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:45.100125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60a901-d7f5-473d-ae0e-d5d570ec8942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:45.100125 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60a901-d7f5-473d-ae0e-d5d570ec8942 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:45.100600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60a901-d7f5-473d-ae0e-d5d570ec8942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:45.101280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60a901-d7f5-473d-ae0e-d5d570ec8942 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:45.132233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809caf36-eeeb-4dfb-b66d-3c2af61cca7d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:45.133447 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-809caf36-eeeb-4dfb-b66d-3c2af61cca7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:45.149131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809caf36-eeeb-4dfb-b66d-3c2af61cca7d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:45.155235 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bd182-f254-4b72-b1ae-2749b762103a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:45.155882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bd182-f254-4b72-b1ae-2749b762103a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:45.156198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bd182-f254-4b72-b1ae-2749b762103a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:45.156476 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bd182-f254-4b72-b1ae-2749b762103a admin admin] PATH_INFO: `/v3/users/bd535571fee44e17ba71c0f2d75b0aa7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:45.157698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bd182-f254-4b72-b1ae-2749b762103a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd535571fee44e17ba71c0f2d75b0aa7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:45.160021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bd182-f254-4b72-b1ae-2749b762103a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:45.247371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1373/2729] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:45 2022] POST /identity/v3/projects => generated 383 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:45.251256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60a901-d7f5-473d-ae0e-d5d570ec8942 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:45.312491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60a901-d7f5-473d-ae0e-d5d570ec8942 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:45.317229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be97cb0-2d1f-43ee-821d-2d841716077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:45.317980 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be97cb0-2d1f-43ee-821d-2d841716077e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:45.318245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be97cb0-2d1f-43ee-821d-2d841716077e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:45.318524 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be97cb0-2d1f-43ee-821d-2d841716077e admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:45.319191 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be97cb0-2d1f-43ee-821d-2d841716077e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:45.321914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be97cb0-2d1f-43ee-821d-2d841716077e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:45.334356 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1e7bd182-f254-4b72-b1ae-2749b762103a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd535571fee44e17ba71c0f2d75b0aa7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:45.335051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1e7bd182-f254-4b72-b1ae-2749b762103a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd535571fee44e17ba71c0f2d75b0aa7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:45.335674 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1e7bd182-f254-4b72-b1ae-2749b762103a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd535571fee44e17ba71c0f2d75b0aa7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:45.348071 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1e7bd182-f254-4b72-b1ae-2749b762103a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd535571fee44e17ba71c0f2d75b0aa7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:45.414136 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1374/2730] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:45 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:45.418691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be97cb0-2d1f-43ee-821d-2d841716077e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:45.418691 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5be97cb0-2d1f-43ee-821d-2d841716077e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:45.430462 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be97cb0-2d1f-43ee-821d-2d841716077e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:45.435428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d672e0-806f-4936-a086-a8c35f597641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:45.436102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d672e0-806f-4936-a086-a8c35f597641 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:45.436392 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d672e0-806f-4936-a086-a8c35f597641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:45.436668 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d672e0-806f-4936-a086-a8c35f597641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:45.437356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d672e0-806f-4936-a086-a8c35f597641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:45.439012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d672e0-806f-4936-a086-a8c35f597641 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:45.439879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1e7bd182-f254-4b72-b1ae-2749b762103a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd535571fee44e17ba71c0f2d75b0aa7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:45.450877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1375/2731] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:45.455393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d672e0-806f-4936-a086-a8c35f597641 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:45.458836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1357/2732] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:45 2022] DELETE /identity/v3/users/bd535571fee44e17ba71c0f2d75b0aa7 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:45.460291 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f92500-56df-4c13-93e2-6edccefe7809 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:45.460689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f92500-56df-4c13-93e2-6edccefe7809 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:45.461069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f92500-56df-4c13-93e2-6edccefe7809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:45.466823 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d672e0-806f-4936-a086-a8c35f597641 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:45.482548 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-81f92500-56df-4c13-93e2-6edccefe7809 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:45.483846 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-81f92500-56df-4c13-93e2-6edccefe7809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:45.509358 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:45.526447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5408a25-9926-4f68-8afe-71ce9c515896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:45.527117 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5408a25-9926-4f68-8afe-71ce9c515896 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:45.527419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5408a25-9926-4f68-8afe-71ce9c515896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:45.527713 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5408a25-9926-4f68-8afe-71ce9c515896 admin admin] PATH_INFO: `/v3/projects/99070c357a3c44418419926ebe34aced/users/c0317241f3cb4e32bb4f89b5e97f5666/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:45.543645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5408a25-9926-4f68-8afe-71ce9c515896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99070c357a3c44418419926ebe34aced, user_id=c0317241f3cb4e32bb4f89b5e97f5666, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:45.546534 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5408a25-9926-4f68-8afe-71ce9c515896 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:45.550678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1358/2733] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:45.558122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f92500-56df-4c13-93e2-6edccefe7809 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:45.559541 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-81f92500-56df-4c13-93e2-6edccefe7809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:45.612526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:45.616181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f92500-56df-4c13-93e2-6edccefe7809 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:45.619928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c1500-0e82-4011-8f80-5c4ec5be5f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:45.620651 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c1500-0e82-4011-8f80-5c4ec5be5f25 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:45.620933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c1500-0e82-4011-8f80-5c4ec5be5f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:45.621209 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c1500-0e82-4011-8f80-5c4ec5be5f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:45.630398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c1500-0e82-4011-8f80-5c4ec5be5f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:45.631862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c1500-0e82-4011-8f80-5c4ec5be5f25 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:45.636827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1376/2734] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:45 2022] PUT /identity/v3/projects/99070c357a3c44418419926ebe34aced/users/c0317241f3cb4e32bb4f89b5e97f5666/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:45.637610 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5408a25-9926-4f68-8afe-71ce9c515896 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:45.643466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1359/2735] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:49:45.706526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5408a25-9926-4f68-8afe-71ce9c515896 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:45.711409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3345e-aa2e-4197-ac58-018112db66ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:45.712070 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3345e-aa2e-4197-ac58-018112db66ba admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:45.712361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3345e-aa2e-4197-ac58-018112db66ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:45.712651 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3345e-aa2e-4197-ac58-018112db66ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:45.713509 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3345e-aa2e-4197-ac58-018112db66ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:45.715109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3345e-aa2e-4197-ac58-018112db66ba admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:45.729585 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1377/2736] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:45.731184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c1500-0e82-4011-8f80-5c4ec5be5f25 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:45.741831 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c1500-0e82-4011-8f80-5c4ec5be5f25 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:45.748158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce45c6-6921-4209-96f9-fc73a362d131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:45.748841 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce45c6-6921-4209-96f9-fc73a362d131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:45.749134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce45c6-6921-4209-96f9-fc73a362d131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:45.749935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce45c6-6921-4209-96f9-fc73a362d131 admin admin] PATH_INFO: `/v3/projects/99070c357a3c44418419926ebe34aced/users/c0317241f3cb4e32bb4f89b5e97f5666/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:45.752129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce45c6-6921-4209-96f9-fc73a362d131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99070c357a3c44418419926ebe34aced, user_id=c0317241f3cb4e32bb4f89b5e97f5666, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:45.754834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce45c6-6921-4209-96f9-fc73a362d131 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:45.765164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1360/2737] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:45 2022] PUT /identity/v3/projects/99070c357a3c44418419926ebe34aced/users/c0317241f3cb4e32bb4f89b5e97f5666/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:45.776441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b7dfc-6620-45c4-87a2-7f456542eb18 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:45.776724 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b7dfc-6620-45c4-87a2-7f456542eb18 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:45.777000 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b7dfc-6620-45c4-87a2-7f456542eb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:45.820796 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f93b7dfc-6620-45c4-87a2-7f456542eb18 None None] MFA Rules not processed for user `c0317241f3cb4e32bb4f89b5e97f5666`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:45.822008 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f93b7dfc-6620-45c4-87a2-7f456542eb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:45.898761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1378/2738] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:45.973063 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce45c6-6921-4209-96f9-fc73a362d131 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:45.974996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f9ce45c6-6921-4209-96f9-fc73a362d131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:46.047782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce45c6-6921-4209-96f9-fc73a362d131 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:46.051433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee260b-4f88-4513-8551-32f71a13e7ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:46.052086 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee260b-4f88-4513-8551-32f71a13e7ae service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:46.052372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee260b-4f88-4513-8551-32f71a13e7ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:46.052648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee260b-4f88-4513-8551-32f71a13e7ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:46.054467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7eee260b-4f88-4513-8551-32f71a13e7ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:46.123788 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee260b-4f88-4513-8551-32f71a13e7ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:46.125309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee260b-4f88-4513-8551-32f71a13e7ae service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:46.134803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1361/2739] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:46.281704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b7dfc-6620-45c4-87a2-7f456542eb18 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:46.283187 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f93b7dfc-6620-45c4-87a2-7f456542eb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:46.348500 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:46.364889 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b7dfc-6620-45c4-87a2-7f456542eb18 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:46.371193 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:46.371827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:46.372152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:46.372432 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] PATH_INFO: `/v3/projects/ebc009a20a10430583f8a0a52d8f6903` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:46.373714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc009a20a10430583f8a0a52d8f6903)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:46.375620 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:46.535202 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc009a20a10430583f8a0a52d8f6903', 'request_id': 'req-f965ed6d-671e-4204-8e19-04e3c657b5f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:46.639537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc009a20a10430583f8a0a52d8f6903', 'request_id': 'req-f965ed6d-671e-4204-8e19-04e3c657b5f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:46.645833 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc009a20a10430583f8a0a52d8f6903', 'request_id': 'req-f965ed6d-671e-4204-8e19-04e3c657b5f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:46.657772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] The token cache is being invalidate because project ebc009a20a10430583f8a0a52d8f6903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:46.658091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:46.658730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f965ed6d-671e-4204-8e19-04e3c657b5f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:46.660371 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1379/2740] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:46 2022] DELETE /identity/v3/projects/ebc009a20a10430583f8a0a52d8f6903 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:46.665409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a0f56-2c98-4f42-90d8-08556bf9812b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:46.665888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a0f56-2c98-4f42-90d8-08556bf9812b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:46.666564 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a0f56-2c98-4f42-90d8-08556bf9812b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:46.686973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ef4a0f56-2c98-4f42-90d8-08556bf9812b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:46.688242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ef4a0f56-2c98-4f42-90d8-08556bf9812b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:46.748095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1362/2741] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:46.755895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c92c8-3c7c-4471-a049-43b60bf110ef None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:46.756250 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c92c8-3c7c-4471-a049-43b60bf110ef None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:46.756532 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c92c8-3c7c-4471-a049-43b60bf110ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:46.776503 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-702c92c8-3c7c-4471-a049-43b60bf110ef None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:46.777706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-702c92c8-3c7c-4471-a049-43b60bf110ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:46.786837 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1380/2742] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:46.789806 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a0f56-2c98-4f42-90d8-08556bf9812b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:46.800422 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a0f56-2c98-4f42-90d8-08556bf9812b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:46.805024 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de260a-3031-40de-8271-271fafd3d2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:46.805662 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de260a-3031-40de-8271-271fafd3d2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:46.805949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de260a-3031-40de-8271-271fafd3d2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:46.806229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de260a-3031-40de-8271-271fafd3d2b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:46.807848 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de260a-3031-40de-8271-271fafd3d2b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:49:46.808187 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de260a-3031-40de-8271-271fafd3d2b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:46.809789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de260a-3031-40de-8271-271fafd3d2b9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:46.821373 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1363/2743] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:49:46.825848 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c92c8-3c7c-4471-a049-43b60bf110ef None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:46.835426 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c92c8-3c7c-4471-a049-43b60bf110ef None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:46.840887 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace98d0-7fa7-4963-91a6-0a33cdcf89cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:46.841507 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace98d0-7fa7-4963-91a6-0a33cdcf89cc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:46.841788 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace98d0-7fa7-4963-91a6-0a33cdcf89cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:46.842063 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace98d0-7fa7-4963-91a6-0a33cdcf89cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:46.842804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace98d0-7fa7-4963-91a6-0a33cdcf89cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:46.844787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace98d0-7fa7-4963-91a6-0a33cdcf89cc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:46.932663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1381/2744] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:46 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:46.936923 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de260a-3031-40de-8271-271fafd3d2b9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:46.952101 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de260a-3031-40de-8271-271fafd3d2b9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:46.972886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:47.012051 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e78db4-b4f0-4ee0-9b1c-35e4deeb8aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:47.012683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78db4-b4f0-4ee0-9b1c-35e4deeb8aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:47.012970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78db4-b4f0-4ee0-9b1c-35e4deeb8aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:47.013251 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78db4-b4f0-4ee0-9b1c-35e4deeb8aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:47.014010 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e78db4-b4f0-4ee0-9b1c-35e4deeb8aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:47.015935 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e78db4-b4f0-4ee0-9b1c-35e4deeb8aeb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:47.101350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1364/2745] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:46 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:47.106712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace98d0-7fa7-4963-91a6-0a33cdcf89cc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:47.117101 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace98d0-7fa7-4963-91a6-0a33cdcf89cc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:47.122127 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a63f7-d486-4da1-bc58-c66c1b7f717d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:47.122735 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a63f7-d486-4da1-bc58-c66c1b7f717d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:47.123016 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a63f7-d486-4da1-bc58-c66c1b7f717d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:47.123289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a63f7-d486-4da1-bc58-c66c1b7f717d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:47.124035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a63f7-d486-4da1-bc58-c66c1b7f717d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:47.125663 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a63f7-d486-4da1-bc58-c66c1b7f717d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:47.137203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1382/2746] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:47.143533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e78db4-b4f0-4ee0-9b1c-35e4deeb8aeb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:47.154579 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e78db4-b4f0-4ee0-9b1c-35e4deeb8aeb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:47.159497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81b554-5737-4316-aae2-c22721c69b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:47.160170 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81b554-5737-4316-aae2-c22721c69b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:47.160464 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81b554-5737-4316-aae2-c22721c69b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:47.160739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81b554-5737-4316-aae2-c22721c69b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:47.161404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81b554-5737-4316-aae2-c22721c69b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:47.162994 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81b554-5737-4316-aae2-c22721c69b17 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:47.175048 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1365/2747] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:47.181311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a63f7-d486-4da1-bc58-c66c1b7f717d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:47.204652 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a63f7-d486-4da1-bc58-c66c1b7f717d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:47.207561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5413307-f866-484b-8a04-9090705ec852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:47.208839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5413307-f866-484b-8a04-9090705ec852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:47.209494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5413307-f866-484b-8a04-9090705ec852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:47.210112 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5413307-f866-484b-8a04-9090705ec852 admin admin] PATH_INFO: `/v3/projects/656e541313a049e788238fbb08cd02c4/users/5231bfebf4bf493baa62fb2ab899de53/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:47.244774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5413307-f866-484b-8a04-9090705ec852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=656e541313a049e788238fbb08cd02c4, user_id=5231bfebf4bf493baa62fb2ab899de53, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:47.250345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5413307-f866-484b-8a04-9090705ec852 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:47.349046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1383/2748] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:47 2022] PUT /identity/v3/projects/656e541313a049e788238fbb08cd02c4/users/5231bfebf4bf493baa62fb2ab899de53/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:47.359775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3b48a-0411-4888-833c-c63a03654809 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:47.360091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3b48a-0411-4888-833c-c63a03654809 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:47.360373 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3b48a-0411-4888-833c-c63a03654809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:47.392053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c1e3b48a-0411-4888-833c-c63a03654809 None None] MFA Rules not processed for user `5231bfebf4bf493baa62fb2ab899de53`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:47.393401 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c1e3b48a-0411-4888-833c-c63a03654809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:47.461620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1366/2749] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:47.473242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5413307-f866-484b-8a04-9090705ec852 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:47.474923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b5413307-f866-484b-8a04-9090705ec852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:47.543378 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:47.575059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5413307-f866-484b-8a04-9090705ec852 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:47.581428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cec0d2-f045-4c3b-9d7c-e7d8271f7eaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:47.582039 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cec0d2-f045-4c3b-9d7c-e7d8271f7eaf service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:47.582321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cec0d2-f045-4c3b-9d7c-e7d8271f7eaf service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:47.582597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cec0d2-f045-4c3b-9d7c-e7d8271f7eaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:47.591059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cec0d2-f045-4c3b-9d7c-e7d8271f7eaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:47.592693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cec0d2-f045-4c3b-9d7c-e7d8271f7eaf service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:47.601248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1384/2750] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:47.809719 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3b48a-0411-4888-833c-c63a03654809 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:47.871439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3b48a-0411-4888-833c-c63a03654809 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:47.875856 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:47.876523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:47.876810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:47.877089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] PATH_INFO: `/v3/users/5231bfebf4bf493baa62fb2ab899de53` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:47.878255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5231bfebf4bf493baa62fb2ab899de53)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:47.880200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:48.011515 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:48.024194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5231bfebf4bf493baa62fb2ab899de53'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:48.024843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5231bfebf4bf493baa62fb2ab899de53'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:48.025563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5231bfebf4bf493baa62fb2ab899de53'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:48.037439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5231bfebf4bf493baa62fb2ab899de53'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:48.130484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5231bfebf4bf493baa62fb2ab899de53'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:48.143573 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1367/2751] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:49:47 2022] DELETE /identity/v3/users/5231bfebf4bf493baa62fb2ab899de53 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:48.148128 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ed6e9-e0ce-4a86-aebd-79a294cdef83 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:48.148422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ed6e9-e0ce-4a86-aebd-79a294cdef83 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:48.148704 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ed6e9-e0ce-4a86-aebd-79a294cdef83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:48.168341 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2e1ed6e9-e0ce-4a86-aebd-79a294cdef83 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:48.169583 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2e1ed6e9-e0ce-4a86-aebd-79a294cdef83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:48.228256 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1385/2752] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:48.238638 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:48.240069 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:48.296388 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4791143-985a-4cdb-8744-3cdba4c30d86 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:48.300015 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430bd8cc-ddd7-4814-8131-de5802d1f298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:48.300631 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bd8cc-ddd7-4814-8131-de5802d1f298 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:48.300913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bd8cc-ddd7-4814-8131-de5802d1f298 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:48.301191 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bd8cc-ddd7-4814-8131-de5802d1f298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:48.310764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bd8cc-ddd7-4814-8131-de5802d1f298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:48.312228 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bd8cc-ddd7-4814-8131-de5802d1f298 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:48.321508 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1368/2753] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:48.744400 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ed6e9-e0ce-4a86-aebd-79a294cdef83 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:48.746060 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2e1ed6e9-e0ce-4a86-aebd-79a294cdef83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:48.758991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ed6e9-e0ce-4a86-aebd-79a294cdef83 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:48.764674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:48.765723 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:48.765723 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:48.766005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] PATH_INFO: `/v3/projects/656e541313a049e788238fbb08cd02c4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:48.767123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=656e541313a049e788238fbb08cd02c4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:48.769307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:48.878781 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '656e541313a049e788238fbb08cd02c4', 'request_id': 'req-86ebbd1c-b019-480f-82c8-d088c44e05d7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:48.936276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '656e541313a049e788238fbb08cd02c4', 'request_id': 'req-86ebbd1c-b019-480f-82c8-d088c44e05d7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:48.942550 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '656e541313a049e788238fbb08cd02c4', 'request_id': 'req-86ebbd1c-b019-480f-82c8-d088c44e05d7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:48.954711 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] The token cache is being invalidate because project 656e541313a049e788238fbb08cd02c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:49:48.955178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:48.955831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86ebbd1c-b019-480f-82c8-d088c44e05d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:49:48.957926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1386/2754] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:49:48 2022] DELETE /identity/v3/projects/656e541313a049e788238fbb08cd02c4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:48.963859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4caeb7-12a4-483e-ae34-6bc753a59fa8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:48.964321 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4caeb7-12a4-483e-ae34-6bc753a59fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:48.964861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4caeb7-12a4-483e-ae34-6bc753a59fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:48.986358 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fa4caeb7-12a4-483e-ae34-6bc753a59fa8 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:48.987526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fa4caeb7-12a4-483e-ae34-6bc753a59fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:49.059533 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1369/2755] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:49:49.069011 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fca90-98de-462a-80a8-1692ef55cdd9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:49.069353 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fca90-98de-462a-80a8-1692ef55cdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:49.069593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fca90-98de-462a-80a8-1692ef55cdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:49.091748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-445fca90-98de-462a-80a8-1692ef55cdd9 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:49.092981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-445fca90-98de-462a-80a8-1692ef55cdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:49.100998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1387/2756] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:49.105873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4caeb7-12a4-483e-ae34-6bc753a59fa8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:49.115894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4caeb7-12a4-483e-ae34-6bc753a59fa8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:49.121907 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19396998-7841-4fde-be43-049135415251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:49.122689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19396998-7841-4fde-be43-049135415251 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:49.122978 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19396998-7841-4fde-be43-049135415251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:49.123260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19396998-7841-4fde-be43-049135415251 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:49.123967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19396998-7841-4fde-be43-049135415251 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:49:49.124301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19396998-7841-4fde-be43-049135415251 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:49.125937 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19396998-7841-4fde-be43-049135415251 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:49.138754 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1370/2757] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:49:49.143207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fca90-98de-462a-80a8-1692ef55cdd9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:49.153130 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fca90-98de-462a-80a8-1692ef55cdd9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:49.157652 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be3c08-755d-404b-a0c9-29b65f724961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:49.158254 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be3c08-755d-404b-a0c9-29b65f724961 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:49.158529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be3c08-755d-404b-a0c9-29b65f724961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:49.158802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be3c08-755d-404b-a0c9-29b65f724961 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:49.159456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be3c08-755d-404b-a0c9-29b65f724961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:49.161428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be3c08-755d-404b-a0c9-29b65f724961 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:49.300147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1388/2758] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:49 2022] POST /identity/v3/projects => generated 407 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:49.303710 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19396998-7841-4fde-be43-049135415251 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:49.314414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19396998-7841-4fde-be43-049135415251 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:49.349925 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:49.373435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e9a08-11dd-4070-a65f-b2cdfe9e2ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:49.374088 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e9a08-11dd-4070-a65f-b2cdfe9e2ade admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:49.374391 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e9a08-11dd-4070-a65f-b2cdfe9e2ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:49.374678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e9a08-11dd-4070-a65f-b2cdfe9e2ade admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:49.375407 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e9a08-11dd-4070-a65f-b2cdfe9e2ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:49.377501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e9a08-11dd-4070-a65f-b2cdfe9e2ade admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:49.434036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1371/2759] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:49 2022] POST /identity/v3/users => generated 304 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:49.437796 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be3c08-755d-404b-a0c9-29b65f724961 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:49.448608 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be3c08-755d-404b-a0c9-29b65f724961 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:49.454556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa9863-1dec-4fa1-b121-51014ca5d219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:49.455284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa9863-1dec-4fa1-b121-51014ca5d219 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:49.455584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa9863-1dec-4fa1-b121-51014ca5d219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:49.455856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa9863-1dec-4fa1-b121-51014ca5d219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:49.456572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa9863-1dec-4fa1-b121-51014ca5d219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:49.458158 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa9863-1dec-4fa1-b121-51014ca5d219 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:49.469517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1389/2760] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:49.473697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e9a08-11dd-4070-a65f-b2cdfe9e2ade admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:49.483647 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e9a08-11dd-4070-a65f-b2cdfe9e2ade admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:49.488354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226d448-3c08-44e3-aff0-577305c12ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:49.488981 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226d448-3c08-44e3-aff0-577305c12ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:49.489532 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226d448-3c08-44e3-aff0-577305c12ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:49.489884 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226d448-3c08-44e3-aff0-577305c12ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:49.490610 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226d448-3c08-44e3-aff0-577305c12ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:49.492862 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226d448-3c08-44e3-aff0-577305c12ef2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:49.503683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1372/2761] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:49.507902 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa9863-1dec-4fa1-b121-51014ca5d219 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:49.518292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa9863-1dec-4fa1-b121-51014ca5d219 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:49.523402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd055cf9-4607-46ed-b996-7adc0b180cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:49.524091 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd055cf9-4607-46ed-b996-7adc0b180cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:49.524377 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd055cf9-4607-46ed-b996-7adc0b180cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:49.524655 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd055cf9-4607-46ed-b996-7adc0b180cc1 admin admin] PATH_INFO: `/v3/projects/92a56ef4f1324058951451802c00091d/users/95560492b19e4e808a99cb8b067e2d10/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:49.537804 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:49.541292 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd055cf9-4607-46ed-b996-7adc0b180cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a56ef4f1324058951451802c00091d, user_id=95560492b19e4e808a99cb8b067e2d10, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:49.544348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd055cf9-4607-46ed-b996-7adc0b180cc1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:49.638912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1390/2762] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:49 2022] PUT /identity/v3/projects/92a56ef4f1324058951451802c00091d/users/95560492b19e4e808a99cb8b067e2d10/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:49:49.649492 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a9ded-d293-4801-acb3-4c00cf3a4972 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:49.649865 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a9ded-d293-4801-acb3-4c00cf3a4972 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:49.650120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a9ded-d293-4801-acb3-4c00cf3a4972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:49.681129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cd3a9ded-d293-4801-acb3-4c00cf3a4972 None None] MFA Rules not processed for user `95560492b19e4e808a99cb8b067e2d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:49.682365 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cd3a9ded-d293-4801-acb3-4c00cf3a4972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:49.746658 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1373/2763] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:49 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:49.751175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd055cf9-4607-46ed-b996-7adc0b180cc1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:49.812472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd055cf9-4607-46ed-b996-7adc0b180cc1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:49.817152 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5f08e-bfa4-4d4d-99b6-e754a08a03a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:49.817762 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5f08e-bfa4-4d4d-99b6-e754a08a03a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:49.818042 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5f08e-bfa4-4d4d-99b6-e754a08a03a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:49.818315 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5f08e-bfa4-4d4d-99b6-e754a08a03a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:49.819014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5f08e-bfa4-4d4d-99b6-e754a08a03a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:49.820962 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5f08e-bfa4-4d4d-99b6-e754a08a03a2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:49.932336 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1391/2764] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:49:49 2022] POST /identity/v3/projects => generated 407 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:49:49.937249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a9ded-d293-4801-acb3-4c00cf3a4972 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:49.947843 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a9ded-d293-4801-acb3-4c00cf3a4972 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:50.009537 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6317874-a404-4276-ab30-8483194d0d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:50.010252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6317874-a404-4276-ab30-8483194d0d20 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:50.010437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6317874-a404-4276-ab30-8483194d0d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:50.010741 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6317874-a404-4276-ab30-8483194d0d20 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:50.011446 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6317874-a404-4276-ab30-8483194d0d20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:50.014280 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6317874-a404-4276-ab30-8483194d0d20 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:50.139432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1374/2765] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:49:49 2022] POST /identity/v3/users => generated 310 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:49:50.144994 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5f08e-bfa4-4d4d-99b6-e754a08a03a2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:50.155529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5f08e-bfa4-4d4d-99b6-e754a08a03a2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:50.160604 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeee9e4-2670-4967-8142-3d129b7d9a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:50.161508 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeee9e4-2670-4967-8142-3d129b7d9a0d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:50.161802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeee9e4-2670-4967-8142-3d129b7d9a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:50.162109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeee9e4-2670-4967-8142-3d129b7d9a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:50.162934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeee9e4-2670-4967-8142-3d129b7d9a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:50.164688 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeee9e4-2670-4967-8142-3d129b7d9a0d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:50.176910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1392/2766] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:49:50.181156 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6317874-a404-4276-ab30-8483194d0d20 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:50.200053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6317874-a404-4276-ab30-8483194d0d20 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:50.200053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef196-9598-4f11-9211-ae87057c51ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:50.200053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef196-9598-4f11-9211-ae87057c51ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:50.200053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef196-9598-4f11-9211-ae87057c51ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:50.200847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef196-9598-4f11-9211-ae87057c51ab admin admin] PATH_INFO: `/v3/projects/13bc189ee5334b1ba6edd5214a8ed2e3/users/d3206789fd7d449387c7a78e5d424000/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:50.218837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ef196-9598-4f11-9211-ae87057c51ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bc189ee5334b1ba6edd5214a8ed2e3, user_id=d3206789fd7d449387c7a78e5d424000, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:50.221773 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ef196-9598-4f11-9211-ae87057c51ab admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:50.309212 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeee9e4-2670-4967-8142-3d129b7d9a0d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:50.311597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6aeee9e4-2670-4967-8142-3d129b7d9a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:50.315158 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1375/2767] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:50 2022] PUT /identity/v3/projects/13bc189ee5334b1ba6edd5214a8ed2e3/users/d3206789fd7d449387c7a78e5d424000/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:50.322004 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef196-9598-4f11-9211-ae87057c51ab admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:50.335104 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:50.372673 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeee9e4-2670-4967-8142-3d129b7d9a0d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:50.389414 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef196-9598-4f11-9211-ae87057c51ab admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:50.395861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622f48-52c4-43e7-acc8-75818f84ca1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:50.396609 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622f48-52c4-43e7-acc8-75818f84ca1d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:50.396857 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622f48-52c4-43e7-acc8-75818f84ca1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:50.397135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622f48-52c4-43e7-acc8-75818f84ca1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:50.397969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a622f48-52c4-43e7-acc8-75818f84ca1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:50.400011 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a622f48-52c4-43e7-acc8-75818f84ca1d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:50.411845 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1376/2768] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:49:50.416055 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622f48-52c4-43e7-acc8-75818f84ca1d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:50.425684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622f48-52c4-43e7-acc8-75818f84ca1d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:50.427742 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c5079-273e-4f92-868f-d847a3084407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:50.428402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c5079-273e-4f92-868f-d847a3084407 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:50.428690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c5079-273e-4f92-868f-d847a3084407 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:50.428962 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c5079-273e-4f92-868f-d847a3084407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:50.430717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d1957-049c-4cb5-9abe-469ce066c52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:50.430840 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2b5c5079-273e-4f92-868f-d847a3084407 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:50.431372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d1957-049c-4cb5-9abe-469ce066c52e admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:50.431657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d1957-049c-4cb5-9abe-469ce066c52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:50.431936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d1957-049c-4cb5-9abe-469ce066c52e admin admin] PATH_INFO: `/v3/projects/13bc189ee5334b1ba6edd5214a8ed2e3/users/d3206789fd7d449387c7a78e5d424000/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:50.433928 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d1957-049c-4cb5-9abe-469ce066c52e admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bc189ee5334b1ba6edd5214a8ed2e3, user_id=d3206789fd7d449387c7a78e5d424000, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:50.439917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d1957-049c-4cb5-9abe-469ce066c52e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:50.448288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1377/2769] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:49:50 2022] PUT /identity/v3/projects/13bc189ee5334b1ba6edd5214a8ed2e3/users/d3206789fd7d449387c7a78e5d424000/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:49:50.459578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c5e23-0d5e-47ac-ac5c-4a649215e1f6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:50.459859 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c5e23-0d5e-47ac-ac5c-4a649215e1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:50.460168 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c5e23-0d5e-47ac-ac5c-4a649215e1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:50.467258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:50.488920 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f14c5e23-0d5e-47ac-ac5c-4a649215e1f6 None None] MFA Rules not processed for user `d3206789fd7d449387c7a78e5d424000`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:49:50.490118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f14c5e23-0d5e-47ac-ac5c-4a649215e1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:50.491242 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c5079-273e-4f92-868f-d847a3084407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:50.492730 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c5079-273e-4f92-868f-d847a3084407 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:50.501698 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1393/2770] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:50.573739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1378/2771] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:49:50 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:49:50.596303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c5079-273e-4f92-868f-d847a3084407 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:50.597909 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2b5c5079-273e-4f92-868f-d847a3084407 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:50.609477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c5079-273e-4f92-868f-d847a3084407 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:50.659012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9548f-fda5-4311-92c3-d84076652664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:50.659660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9548f-fda5-4311-92c3-d84076652664 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:50.659943 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9548f-fda5-4311-92c3-d84076652664 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:50.660248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9548f-fda5-4311-92c3-d84076652664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:50.670757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9548f-fda5-4311-92c3-d84076652664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:50.672261 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9548f-fda5-4311-92c3-d84076652664 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:50.681618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1394/2772] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:50 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:51.111136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c5e23-0d5e-47ac-ac5c-4a649215e1f6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:51.121255 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c5e23-0d5e-47ac-ac5c-4a649215e1f6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:51.124888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4de558-8aa0-438a-bbe0-2b2d80d3665e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:51.125498 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4de558-8aa0-438a-bbe0-2b2d80d3665e service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:51.125781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4de558-8aa0-438a-bbe0-2b2d80d3665e service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:51.126053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4de558-8aa0-438a-bbe0-2b2d80d3665e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:51.181578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4de558-8aa0-438a-bbe0-2b2d80d3665e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:51.183075 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4de558-8aa0-438a-bbe0-2b2d80d3665e service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:51.191867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1379/2773] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:51 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:54.806372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9548f-fda5-4311-92c3-d84076652664 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:54.806372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a8e9548f-fda5-4311-92c3-d84076652664 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:54.875029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4de558-8aa0-438a-bbe0-2b2d80d3665e service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:49:54.884119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4de558-8aa0-438a-bbe0-2b2d80d3665e service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:54.888334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efeb5f7-f156-4085-bc3a-e9bd0f1cb3ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:54.892518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efeb5f7-f156-4085-bc3a-e9bd0f1cb3ec service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:54.892518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efeb5f7-f156-4085-bc3a-e9bd0f1cb3ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:54.892518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efeb5f7-f156-4085-bc3a-e9bd0f1cb3ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:54.892518 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9efeb5f7-f156-4085-bc3a-e9bd0f1cb3ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:54.902396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9548f-fda5-4311-92c3-d84076652664 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:49:54.906968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:49:54.907740 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0932e8-1ea7-4bd2-a82c-8a03867c0028 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:49:54.908444 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0932e8-1ea7-4bd2-a82c-8a03867c0028 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:49:54.908791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0932e8-1ea7-4bd2-a82c-8a03867c0028 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:49:54.909131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0932e8-1ea7-4bd2-a82c-8a03867c0028 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:49:54.910862 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6a0932e8-1ea7-4bd2-a82c-8a03867c0028 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:49:54.930991 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0932e8-1ea7-4bd2-a82c-8a03867c0028 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:54.932732 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0932e8-1ea7-4bd2-a82c-8a03867c0028 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:54.945526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1395/2774] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:49:54.975022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efeb5f7-f156-4085-bc3a-e9bd0f1cb3ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:49:54.976538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efeb5f7-f156-4085-bc3a-e9bd0f1cb3ec service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:49:54.987023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1380/2775] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:49:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:50:04.899935 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0932e8-1ea7-4bd2-a82c-8a03867c0028 service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:04.949502 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:04.964307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0932e8-1ea7-4bd2-a82c-8a03867c0028 service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:04.969790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d21e4-0744-4c7f-9c83-b40251f21975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:04.970439 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d21e4-0744-4c7f-9c83-b40251f21975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:04.970726 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d21e4-0744-4c7f-9c83-b40251f21975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:04.971013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d21e4-0744-4c7f-9c83-b40251f21975 admin admin] PATH_INFO: `/v3/users/95560492b19e4e808a99cb8b067e2d10` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:04.972195 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d21e4-0744-4c7f-9c83-b40251f21975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95560492b19e4e808a99cb8b067e2d10)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:04.974216 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d21e4-0744-4c7f-9c83-b40251f21975 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:05.159538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efeb5f7-f156-4085-bc3a-e9bd0f1cb3ec service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:05.161049 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9efeb5f7-f156-4085-bc3a-e9bd0f1cb3ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:05.174471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efeb5f7-f156-4085-bc3a-e9bd0f1cb3ec service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:05.179154 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3194f3-dec3-493b-ba4b-99cde09eb185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:05.179657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3194f3-dec3-493b-ba4b-99cde09eb185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:05.179942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3194f3-dec3-493b-ba4b-99cde09eb185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:05.180317 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3194f3-dec3-493b-ba4b-99cde09eb185 admin admin] PATH_INFO: `/v3/users/e931214042b542ceb969ffea0cf41560` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:05.181543 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3194f3-dec3-493b-ba4b-99cde09eb185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e931214042b542ceb969ffea0cf41560)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:05.183457 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3194f3-dec3-493b-ba4b-99cde09eb185 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:05.213473 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c76d21e4-0744-4c7f-9c83-b40251f21975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95560492b19e4e808a99cb8b067e2d10'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.214188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c76d21e4-0744-4c7f-9c83-b40251f21975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95560492b19e4e808a99cb8b067e2d10'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.221679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c76d21e4-0744-4c7f-9c83-b40251f21975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95560492b19e4e808a99cb8b067e2d10'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.233215 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c76d21e4-0744-4c7f-9c83-b40251f21975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95560492b19e4e808a99cb8b067e2d10'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.319025 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c76d21e4-0744-4c7f-9c83-b40251f21975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95560492b19e4e808a99cb8b067e2d10'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.325243 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1396/2776] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:50:04 2022] DELETE /identity/v3/users/95560492b19e4e808a99cb8b067e2d10 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:05.330386 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cf7da-d3d3-4315-936b-e063fba6c839 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:05.330672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cf7da-d3d3-4315-936b-e063fba6c839 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:05.330950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cf7da-d3d3-4315-936b-e063fba6c839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:05.351304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ff1cf7da-d3d3-4315-936b-e063fba6c839 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:05.351304 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ff1cf7da-d3d3-4315-936b-e063fba6c839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:05.384249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c3194f3-dec3-493b-ba4b-99cde09eb185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e931214042b542ceb969ffea0cf41560'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.385018 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c3194f3-dec3-493b-ba4b-99cde09eb185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e931214042b542ceb969ffea0cf41560'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.385622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c3194f3-dec3-493b-ba4b-99cde09eb185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e931214042b542ceb969ffea0cf41560'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.399734 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c3194f3-dec3-493b-ba4b-99cde09eb185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e931214042b542ceb969ffea0cf41560'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.411083 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1397/2777] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:50:05.420756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cf7da-d3d3-4315-936b-e063fba6c839 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:05.422178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ff1cf7da-d3d3-4315-936b-e063fba6c839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:05.467557 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c3194f3-dec3-493b-ba4b-99cde09eb185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e931214042b542ceb969ffea0cf41560'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.481534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1381/2778] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:50:05 2022] DELETE /identity/v3/users/e931214042b542ceb969ffea0cf41560 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:05.485906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c531c0b-fae2-4148-89a7-68a466ca70e6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:05.486197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c531c0b-fae2-4148-89a7-68a466ca70e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:05.488003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c531c0b-fae2-4148-89a7-68a466ca70e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:05.502956 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:05.506184 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cf7da-d3d3-4315-936b-e063fba6c839 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:05.517559 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9c531c0b-fae2-4148-89a7-68a466ca70e6 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:05.518950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9c531c0b-fae2-4148-89a7-68a466ca70e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:05.556196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b1837-69b3-4064-bafb-a603a3bf98c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:05.556814 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b1837-69b3-4064-bafb-a603a3bf98c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:05.557093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b1837-69b3-4064-bafb-a603a3bf98c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:05.557369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b1837-69b3-4064-bafb-a603a3bf98c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:05.579703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1382/2779] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:50:05.583790 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c531c0b-fae2-4148-89a7-68a466ca70e6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:05.585137 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9c531c0b-fae2-4148-89a7-68a466ca70e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:05.596972 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c531c0b-fae2-4148-89a7-68a466ca70e6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:05.602143 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:05.602724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:05.603009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:05.603287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] PATH_INFO: `/v3/users/da8e9a3148cd4a62988f9cb332289c54` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:05.604513 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=da8e9a3148cd4a62988f9cb332289c54)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:05.606438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:05.620484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b1837-69b3-4064-bafb-a603a3bf98c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:05.622103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b1837-69b3-4064-bafb-a603a3bf98c2 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:05.629305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:05.632917 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1398/2780] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:50:05.633715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b1837-69b3-4064-bafb-a603a3bf98c2 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:05.642716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b1837-69b3-4064-bafb-a603a3bf98c2 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:05.646509 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b5f63-97f8-4157-9bfc-28f4c6a7f833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:05.647109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b5f63-97f8-4157-9bfc-28f4c6a7f833 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:05.647394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b5f63-97f8-4157-9bfc-28f4c6a7f833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:05.647671 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b5f63-97f8-4157-9bfc-28f4c6a7f833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:05.657120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b5f63-97f8-4157-9bfc-28f4c6a7f833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:05.658553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b5f63-97f8-4157-9bfc-28f4c6a7f833 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:05.668753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1399/2781] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:50:05.834607 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da8e9a3148cd4a62988f9cb332289c54'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.835261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da8e9a3148cd4a62988f9cb332289c54'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.835872 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da8e9a3148cd4a62988f9cb332289c54'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.847654 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da8e9a3148cd4a62988f9cb332289c54'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.936172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da8e9a3148cd4a62988f9cb332289c54'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:05.958732 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1383/2782] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:50:05 2022] DELETE /identity/v3/users/da8e9a3148cd4a62988f9cb332289c54 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:05.963385 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ddac8-016c-4011-80d3-d99966d51145 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:05.963778 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ddac8-016c-4011-80d3-d99966d51145 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:05.963946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ddac8-016c-4011-80d3-d99966d51145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:05.984237 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a00ddac8-016c-4011-80d3-d99966d51145 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:05.985424 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a00ddac8-016c-4011-80d3-d99966d51145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:06.046754 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1400/2783] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:06.057302 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:06.059116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:06.125918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa001d-505d-481a-8f44-effdb71711fb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:06.129521 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35dd58-4b7d-45ae-a3a8-d1946cd78683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:06.130172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35dd58-4b7d-45ae-a3a8-d1946cd78683 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:06.130454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35dd58-4b7d-45ae-a3a8-d1946cd78683 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:06.130731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35dd58-4b7d-45ae-a3a8-d1946cd78683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:06.144343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35dd58-4b7d-45ae-a3a8-d1946cd78683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:06.144343 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35dd58-4b7d-45ae-a3a8-d1946cd78683 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:06.144557 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ddac8-016c-4011-80d3-d99966d51145 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:06.145501 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a00ddac8-016c-4011-80d3-d99966d51145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:06.153197 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1384/2784] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:50:06.161930 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ddac8-016c-4011-80d3-d99966d51145 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:06.167538 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:06.173037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:06.173037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:06.173037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] PATH_INFO: `/v3/projects/92a56ef4f1324058951451802c00091d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:06.173037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a56ef4f1324058951451802c00091d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:06.173037 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:06.200360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35dd58-4b7d-45ae-a3a8-d1946cd78683 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:06.202366 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7d35dd58-4b7d-45ae-a3a8-d1946cd78683 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:06.217585 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35dd58-4b7d-45ae-a3a8-d1946cd78683 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:06.223284 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:06.223998 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:06.224305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:06.224571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] PATH_INFO: `/v3/projects/96ed29d4572a4e47a53cab22abce9f47` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:06.226009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96ed29d4572a4e47a53cab22abce9f47)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:06.227905 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:06.292827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:06.301750 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a56ef4f1324058951451802c00091d', 'request_id': 'req-54781749-ddc3-430d-a679-c48d3aab4779'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.382098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96ed29d4572a4e47a53cab22abce9f47', 'request_id': 'req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.388141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96ed29d4572a4e47a53cab22abce9f47', 'request_id': 'req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.392923 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a56ef4f1324058951451802c00091d', 'request_id': 'req-54781749-ddc3-430d-a679-c48d3aab4779'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.402272 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a56ef4f1324058951451802c00091d', 'request_id': 'req-54781749-ddc3-430d-a679-c48d3aab4779'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.410895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] The token cache is being invalidate because project 92a56ef4f1324058951451802c00091d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:50:06.411192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.412346 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.414033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1401/2785] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:50:06 2022] DELETE /identity/v3/projects/92a56ef4f1324058951451802c00091d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:50:06.417827 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:06.419211 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:06.462301 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96ed29d4572a4e47a53cab22abce9f47', 'request_id': 'req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.475378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] The token cache is being invalidate because project 96ed29d4572a4e47a53cab22abce9f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:50:06.475665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.476377 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.478252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1385/2786] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:50:06 2022] DELETE /identity/v3/projects/96ed29d4572a4e47a53cab22abce9f47 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:50:06.482172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:06.483329 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:06.495159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54781749-ddc3-430d-a679-c48d3aab4779 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:06.511650 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:06.559358 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f92fa5-b5f9-41b9-a535-bb0d32938eb7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:06.560818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:06.561474 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:06.561787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:06.562067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] PATH_INFO: `/v3/users/d3206789fd7d449387c7a78e5d424000` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:06.563251 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3206789fd7d449387c7a78e5d424000)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:06.565204 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:06.565348 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:06.566079 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:06.566437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:06.566727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] PATH_INFO: `/v3/users/66da15ccc1cd451c91629107678ef3d1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:06.567924 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66da15ccc1cd451c91629107678ef3d1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:06.570885 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:06.800700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3206789fd7d449387c7a78e5d424000'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.801370 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3206789fd7d449387c7a78e5d424000'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.808756 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3206789fd7d449387c7a78e5d424000'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.815908 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66da15ccc1cd451c91629107678ef3d1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.816873 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66da15ccc1cd451c91629107678ef3d1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.817776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66da15ccc1cd451c91629107678ef3d1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.821667 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3206789fd7d449387c7a78e5d424000'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.824315 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:06.834055 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66da15ccc1cd451c91629107678ef3d1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.946856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3206789fd7d449387c7a78e5d424000'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:06.954791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1402/2787] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:50:06 2022] DELETE /identity/v3/users/d3206789fd7d449387c7a78e5d424000 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:50:06.955597 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:06.956970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:07.022493 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66da15ccc1cd451c91629107678ef3d1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.027397 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2399a-c041-456e-916b-f5d812eec981 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:07.032676 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:07.033356 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:07.033763 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:07.034046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] PATH_INFO: `/v3/projects/579a54ee385143bfbd37515edd7b8a09` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:07.035357 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=579a54ee385143bfbd37515edd7b8a09)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:07.037425 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:07.042117 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1386/2788] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:50:06 2022] DELETE /identity/v3/users/66da15ccc1cd451c91629107678ef3d1 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:50:07.189518 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '579a54ee385143bfbd37515edd7b8a09', 'request_id': 'req-555c9115-9fa8-4466-99a4-b6f291fa47d5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.268650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '579a54ee385143bfbd37515edd7b8a09', 'request_id': 'req-555c9115-9fa8-4466-99a4-b6f291fa47d5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.283761 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '579a54ee385143bfbd37515edd7b8a09', 'request_id': 'req-555c9115-9fa8-4466-99a4-b6f291fa47d5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.289235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] The token cache is being invalidate because project 579a54ee385143bfbd37515edd7b8a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:50:07.289235 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.289949 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.304280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1403/2789] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:50:06 2022] DELETE /identity/v3/projects/579a54ee385143bfbd37515edd7b8a09 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:07.424575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:07.426138 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:07.500678 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afeb8dd-bc11-4241-b8c6-913098c88ae0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:07.514099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:07.515556 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:07.579134 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:07.592267 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:07.592955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:07.593292 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:07.593659 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] PATH_INFO: `/v3/projects/13bc189ee5334b1ba6edd5214a8ed2e3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:07.594847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13bc189ee5334b1ba6edd5214a8ed2e3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:07.596801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:07.605376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c9115-9fa8-4466-99a4-b6f291fa47d5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:07.611297 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:07.611972 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:07.612321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:07.612603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] PATH_INFO: `/v3/projects/bc193a04208145ae8d5fcdc843435808` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:07.613764 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc193a04208145ae8d5fcdc843435808)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:07.615753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:07.644664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=2,keystone:DELETE=9,keystone:SELECT=89 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:07.707639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13bc189ee5334b1ba6edd5214a8ed2e3', 'request_id': 'req-1fd0d415-70cf-42bd-b5b2-360689098925'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.714148 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13bc189ee5334b1ba6edd5214a8ed2e3', 'request_id': 'req-1fd0d415-70cf-42bd-b5b2-360689098925'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.735381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc193a04208145ae8d5fcdc843435808', 'request_id': 'req-1e71ee13-adf3-48da-853c-d95c2695ffd6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.782693 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13bc189ee5334b1ba6edd5214a8ed2e3', 'request_id': 'req-1fd0d415-70cf-42bd-b5b2-360689098925'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.797680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] The token cache is being invalidate because project 13bc189ee5334b1ba6edd5214a8ed2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:50:07.797980 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.798639 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1fd0d415-70cf-42bd-b5b2-360689098925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.800396 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1387/2790] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:50:07 2022] DELETE /identity/v3/projects/13bc189ee5334b1ba6edd5214a8ed2e3 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:07.807525 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59e64e-4a37-46a6-b720-098b6fea577f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:07.807877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59e64e-4a37-46a6-b720-098b6fea577f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:07.812410 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59e64e-4a37-46a6-b720-098b6fea577f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:07.828007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc193a04208145ae8d5fcdc843435808', 'request_id': 'req-1e71ee13-adf3-48da-853c-d95c2695ffd6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.830246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bd59e64e-4a37-46a6-b720-098b6fea577f None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:07.832253 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bd59e64e-4a37-46a6-b720-098b6fea577f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:07.834369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc193a04208145ae8d5fcdc843435808', 'request_id': 'req-1e71ee13-adf3-48da-853c-d95c2695ffd6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.847347 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] The token cache is being invalidate because project bc193a04208145ae8d5fcdc843435808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:50:07.847640 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.848715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:07.850554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1404/2791] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:50:07 2022] DELETE /identity/v3/projects/bc193a04208145ae8d5fcdc843435808 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:07.855084 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:07.856436 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:07.903281 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1388/2792] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:50:07.913565 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951c6f5-5742-4f9d-a241-0395efd07f5e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:07.913854 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951c6f5-5742-4f9d-a241-0395efd07f5e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:07.914144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951c6f5-5742-4f9d-a241-0395efd07f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:07.925057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71ee13-adf3-48da-853c-d95c2695ffd6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:07.930659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:07.931310 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:07.931594 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:07.931921 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] PATH_INFO: `/v3/users/c0317241f3cb4e32bb4f89b5e97f5666` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:07.933486 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0317241f3cb4e32bb4f89b5e97f5666)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:07.935880 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:07.936929 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8951c6f5-5742-4f9d-a241-0395efd07f5e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:07.938288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8951c6f5-5742-4f9d-a241-0395efd07f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:07.951430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1389/2793] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:50:07.956113 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951c6f5-5742-4f9d-a241-0395efd07f5e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:07.965328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951c6f5-5742-4f9d-a241-0395efd07f5e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:07.970852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e4837-d013-4eed-803d-21e5fb213419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:07.971475 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e4837-d013-4eed-803d-21e5fb213419 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:07.971758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e4837-d013-4eed-803d-21e5fb213419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:07.972064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e4837-d013-4eed-803d-21e5fb213419 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:07.972764 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e4837-d013-4eed-803d-21e5fb213419 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:50:07.973072 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e4837-d013-4eed-803d-21e5fb213419 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:07.974925 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e4837-d013-4eed-803d-21e5fb213419 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:07.989362 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1390/2794] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 07:50:07.994022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e4837-d013-4eed-803d-21e5fb213419 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:08.004258 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e4837-d013-4eed-803d-21e5fb213419 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:08.008895 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bba416-4b20-4db6-9490-b532c86b3e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:08.013378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bba416-4b20-4db6-9490-b532c86b3e38 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:08.013378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bba416-4b20-4db6-9490-b532c86b3e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:08.013378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bba416-4b20-4db6-9490-b532c86b3e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:08.013378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bba416-4b20-4db6-9490-b532c86b3e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:08.013378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bba416-4b20-4db6-9490-b532c86b3e38 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:08.133133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1391/2795] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:50:07 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:50:08.137290 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bba416-4b20-4db6-9490-b532c86b3e38 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:08.158381 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0317241f3cb4e32bb4f89b5e97f5666'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:08.159679 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0317241f3cb4e32bb4f89b5e97f5666'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:08.167481 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0317241f3cb4e32bb4f89b5e97f5666'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:08.180229 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0317241f3cb4e32bb4f89b5e97f5666'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:08.184811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:08.209305 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bba416-4b20-4db6-9490-b532c86b3e38 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:08.215126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1450cf0-ed9d-4cc5-88f1-99a94059940c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:08.216307 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1450cf0-ed9d-4cc5-88f1-99a94059940c admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:08.216545 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1450cf0-ed9d-4cc5-88f1-99a94059940c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:08.216819 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1450cf0-ed9d-4cc5-88f1-99a94059940c admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:08.217728 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1450cf0-ed9d-4cc5-88f1-99a94059940c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:08.221268 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1450cf0-ed9d-4cc5-88f1-99a94059940c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:08.289379 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0317241f3cb4e32bb4f89b5e97f5666'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:08.299362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1405/2796] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:50:07 2022] DELETE /identity/v3/users/c0317241f3cb4e32bb4f89b5e97f5666 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:08.324892 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1392/2797] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:50:08 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:50:08.331433 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:08.333144 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:08.376477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:08.411233 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d8fb1-75ce-4dcb-8751-32b4ac9add25 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:08.416145 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d72a3-462d-4eee-93f8-8253f1e96ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:08.417160 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d72a3-462d-4eee-93f8-8253f1e96ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:08.417666 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d72a3-462d-4eee-93f8-8253f1e96ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:08.418122 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d72a3-462d-4eee-93f8-8253f1e96ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:08.419007 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d72a3-462d-4eee-93f8-8253f1e96ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:08.421305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d72a3-462d-4eee-93f8-8253f1e96ff2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:08.432376 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1406/2798] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:50:08.436562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1450cf0-ed9d-4cc5-88f1-99a94059940c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:08.447029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1450cf0-ed9d-4cc5-88f1-99a94059940c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:08.451809 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14c0a9-c040-4dbe-9a43-9dc3735265f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:08.453006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14c0a9-c040-4dbe-9a43-9dc3735265f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:08.453282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14c0a9-c040-4dbe-9a43-9dc3735265f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:08.453825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14c0a9-c040-4dbe-9a43-9dc3735265f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:08.454780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14c0a9-c040-4dbe-9a43-9dc3735265f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:08.456405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14c0a9-c040-4dbe-9a43-9dc3735265f0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:08.466739 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1393/2799] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:50:08.473514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d72a3-462d-4eee-93f8-8253f1e96ff2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:08.485842 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d72a3-462d-4eee-93f8-8253f1e96ff2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:08.491937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313de10-3057-47fc-8abc-50bf7ac92b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:08.492882 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313de10-3057-47fc-8abc-50bf7ac92b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:08.493162 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313de10-3057-47fc-8abc-50bf7ac92b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:08.494041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313de10-3057-47fc-8abc-50bf7ac92b39 admin admin] PATH_INFO: `/v3/projects/541d2c7940b249fa822e6fcb5e433149/users/0aff8f9d5721484cb58bcc430e165b15/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:08.512466 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313de10-3057-47fc-8abc-50bf7ac92b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=541d2c7940b249fa822e6fcb5e433149, user_id=0aff8f9d5721484cb58bcc430e165b15, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:08.515587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313de10-3057-47fc-8abc-50bf7ac92b39 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:08.588630 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1407/2800] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:50:08 2022] PUT /identity/v3/projects/541d2c7940b249fa822e6fcb5e433149/users/0aff8f9d5721484cb58bcc430e165b15/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:08.593440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6a480-49da-4903-9e12-3f0a5041455e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:08.593820 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6a480-49da-4903-9e12-3f0a5041455e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:08.594166 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6a480-49da-4903-9e12-3f0a5041455e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:08.613648 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a5c6a480-49da-4903-9e12-3f0a5041455e None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:08.615003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a5c6a480-49da-4903-9e12-3f0a5041455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:08.676054 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1394/2801] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:08.685846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313de10-3057-47fc-8abc-50bf7ac92b39 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:08.688151 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f313de10-3057-47fc-8abc-50bf7ac92b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:08.745012 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313de10-3057-47fc-8abc-50bf7ac92b39 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:08.748690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677b2c6-65f6-401d-93ad-c59ea3b304cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:08.749826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677b2c6-65f6-401d-93ad-c59ea3b304cb service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:08.750118 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677b2c6-65f6-401d-93ad-c59ea3b304cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:08.750409 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677b2c6-65f6-401d-93ad-c59ea3b304cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:08.759501 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6a480-49da-4903-9e12-3f0a5041455e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:08.760947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a5c6a480-49da-4903-9e12-3f0a5041455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:08.761174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677b2c6-65f6-401d-93ad-c59ea3b304cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:08.762749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677b2c6-65f6-401d-93ad-c59ea3b304cb service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:08.775779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6a480-49da-4903-9e12-3f0a5041455e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:08.779088 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1408/2802] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:50:08.781006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:08.781777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:08.782112 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:08.782467 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] PATH_INFO: `/v3/projects/99070c357a3c44418419926ebe34aced` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:08.783755 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] RBAC: Authorizing `identity:delete_project(project_id=99070c357a3c44418419926ebe34aced)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:08.786439 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:08.919876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99070c357a3c44418419926ebe34aced', 'request_id': 'req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:08.927013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99070c357a3c44418419926ebe34aced', 'request_id': 'req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:08.986597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99070c357a3c44418419926ebe34aced', 'request_id': 'req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:08.997733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] The token cache is being invalidate because project 99070c357a3c44418419926ebe34aced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:50:08.997960 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:08.998600 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-adb6f507-3f7a-40e8-9a22-d6f5c92ed11e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:09.000760 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1395/2803] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:50:08 2022] DELETE /identity/v3/projects/99070c357a3c44418419926ebe34aced => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:50:09.005545 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5199f-2a16-4108-af48-2037e9855937 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:09.005839 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5199f-2a16-4108-af48-2037e9855937 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:09.006123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5199f-2a16-4108-af48-2037e9855937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:09.025525 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3cb5199f-2a16-4108-af48-2037e9855937 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:09.026782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3cb5199f-2a16-4108-af48-2037e9855937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:09.096131 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1409/2804] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:09.103923 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e813ca-a571-465a-8485-8ea0fd41de98 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:09.104243 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e813ca-a571-465a-8485-8ea0fd41de98 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:09.105240 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e813ca-a571-465a-8485-8ea0fd41de98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:09.125723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-49e813ca-a571-465a-8485-8ea0fd41de98 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:09.126923 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-49e813ca-a571-465a-8485-8ea0fd41de98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:09.135881 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1396/2805] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:09.140099 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5199f-2a16-4108-af48-2037e9855937 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:09.150103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5199f-2a16-4108-af48-2037e9855937 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:09.155036 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f021a0-2f3a-45bc-afe1-9fbf0729bd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:09.155725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f021a0-2f3a-45bc-afe1-9fbf0729bd54 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:09.156085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f021a0-2f3a-45bc-afe1-9fbf0729bd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:09.156415 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f021a0-2f3a-45bc-afe1-9fbf0729bd54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:09.157102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f021a0-2f3a-45bc-afe1-9fbf0729bd54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:50:09.162557 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f021a0-2f3a-45bc-afe1-9fbf0729bd54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:09.162557 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f021a0-2f3a-45bc-afe1-9fbf0729bd54 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:09.170772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1410/2806] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:50:09.175737 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e813ca-a571-465a-8485-8ea0fd41de98 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:09.184919 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e813ca-a571-465a-8485-8ea0fd41de98 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:09.190254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97328b33-55f7-4465-bfb7-9e8017bd9e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:09.190863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97328b33-55f7-4465-bfb7-9e8017bd9e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:09.191150 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97328b33-55f7-4465-bfb7-9e8017bd9e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:09.191430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97328b33-55f7-4465-bfb7-9e8017bd9e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:09.192162 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97328b33-55f7-4465-bfb7-9e8017bd9e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:09.194040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97328b33-55f7-4465-bfb7-9e8017bd9e93 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:09.321667 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1397/2807] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:50:09 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:50:09.322877 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f021a0-2f3a-45bc-afe1-9fbf0729bd54 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:09.333300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f021a0-2f3a-45bc-afe1-9fbf0729bd54 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:09.365220 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:09.392426 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d30c8-866a-4954-9270-7a14415521fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:09.393049 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d30c8-866a-4954-9270-7a14415521fd admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:09.393345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d30c8-866a-4954-9270-7a14415521fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:09.393607 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d30c8-866a-4954-9270-7a14415521fd admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:09.394280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d30c8-866a-4954-9270-7a14415521fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:09.396222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d30c8-866a-4954-9270-7a14415521fd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:09.485772 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1411/2808] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:50:09 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:50:09.489898 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97328b33-55f7-4465-bfb7-9e8017bd9e93 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:09.500144 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97328b33-55f7-4465-bfb7-9e8017bd9e93 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:09.504913 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f90d1-cbc1-496b-a2c9-9ef6c31d46ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:09.505440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f90d1-cbc1-496b-a2c9-9ef6c31d46ae admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:09.505725 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f90d1-cbc1-496b-a2c9-9ef6c31d46ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:09.506009 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f90d1-cbc1-496b-a2c9-9ef6c31d46ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:09.506704 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f90d1-cbc1-496b-a2c9-9ef6c31d46ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:09.508354 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f90d1-cbc1-496b-a2c9-9ef6c31d46ae admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:09.513735 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:09.519487 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1398/2809] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:50:09.523599 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d30c8-866a-4954-9270-7a14415521fd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:09.534700 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d30c8-866a-4954-9270-7a14415521fd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:09.539577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431da0be-c076-4973-aafc-6aa3ab370928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:09.540165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431da0be-c076-4973-aafc-6aa3ab370928 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:09.540571 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431da0be-c076-4973-aafc-6aa3ab370928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:09.540856 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431da0be-c076-4973-aafc-6aa3ab370928 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:09.541548 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431da0be-c076-4973-aafc-6aa3ab370928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:09.543227 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431da0be-c076-4973-aafc-6aa3ab370928 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:09.554108 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1412/2810] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 07:50:09.558309 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f90d1-cbc1-496b-a2c9-9ef6c31d46ae admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:09.568804 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f90d1-cbc1-496b-a2c9-9ef6c31d46ae admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:09.574349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578027d-9bb5-420a-9815-75d8f4af046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:09.575127 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578027d-9bb5-420a-9815-75d8f4af046b admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:09.575496 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578027d-9bb5-420a-9815-75d8f4af046b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:09.575825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578027d-9bb5-420a-9815-75d8f4af046b admin admin] PATH_INFO: `/v3/projects/0ce7dff8f8e54febbd219ec2d9933213/users/2d3d739c143c496f8ba2de5db919cbb5/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:09.592022 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578027d-9bb5-420a-9815-75d8f4af046b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce7dff8f8e54febbd219ec2d9933213, user_id=2d3d739c143c496f8ba2de5db919cbb5, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:09.595587 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578027d-9bb5-420a-9815-75d8f4af046b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:09.689811 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1399/2811] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:50:09 2022] PUT /identity/v3/projects/0ce7dff8f8e54febbd219ec2d9933213/users/2d3d739c143c496f8ba2de5db919cbb5/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:09.699863 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe768-1d1e-48b3-8e0d-f1a682793b82 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:09.700204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe768-1d1e-48b3-8e0d-f1a682793b82 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:09.700493 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe768-1d1e-48b3-8e0d-f1a682793b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:09.729719 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a23fe768-1d1e-48b3-8e0d-f1a682793b82 None None] MFA Rules not processed for user `2d3d739c143c496f8ba2de5db919cbb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:09.730970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a23fe768-1d1e-48b3-8e0d-f1a682793b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:09.798305 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1413/2812] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:09.808744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578027d-9bb5-420a-9815-75d8f4af046b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:09.810933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e578027d-9bb5-420a-9815-75d8f4af046b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:09.891114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578027d-9bb5-420a-9815-75d8f4af046b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:09.895335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac7ec1-8ed0-48a8-8783-80cf60c7fcf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:09.896194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac7ec1-8ed0-48a8-8783-80cf60c7fcf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:09.896656 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac7ec1-8ed0-48a8-8783-80cf60c7fcf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:09.897458 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac7ec1-8ed0-48a8-8783-80cf60c7fcf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:09.906647 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac7ec1-8ed0-48a8-8783-80cf60c7fcf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:09.908118 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac7ec1-8ed0-48a8-8783-80cf60c7fcf9 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:09.917005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1400/2813] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:50:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:50:11.718631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e62fa-c04e-47ac-86f7-461fc2a9a2ad None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:11.719173 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e62fa-c04e-47ac-86f7-461fc2a9a2ad None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:11.719301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e62fa-c04e-47ac-86f7-461fc2a9a2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:11.738790 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-943e62fa-c04e-47ac-86f7-461fc2a9a2ad None None] MFA Rules not processed for user `2d3d739c143c496f8ba2de5db919cbb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:11.740033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-943e62fa-c04e-47ac-86f7-461fc2a9a2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:11.748897 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1414/2814] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:14.039834 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ce1b9-a2a0-4fc9-8117-2a14deefef5a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:14.040412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ce1b9-a2a0-4fc9-8117-2a14deefef5a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:14.040412 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ce1b9-a2a0-4fc9-8117-2a14deefef5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:14.072779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e79ce1b9-a2a0-4fc9-8117-2a14deefef5a None None] MFA Rules not processed for user `0aff8f9d5721484cb58bcc430e165b15`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:14.074106 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e79ce1b9-a2a0-4fc9-8117-2a14deefef5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:14.149216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1401/2815] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:14.153690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e62fa-c04e-47ac-86f7-461fc2a9a2ad None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:14.155965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-943e62fa-c04e-47ac-86f7-461fc2a9a2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:14.239390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e62fa-c04e-47ac-86f7-461fc2a9a2ad None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:14.244749 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b4920-f6d7-43a8-8837-76d5571c9608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:14.245483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b4920-f6d7-43a8-8837-76d5571c9608 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:14.245824 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b4920-f6d7-43a8-8837-76d5571c9608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:14.246033 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b4920-f6d7-43a8-8837-76d5571c9608 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:14.246725 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b4920-f6d7-43a8-8837-76d5571c9608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:14.248894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b4920-f6d7-43a8-8837-76d5571c9608 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:14.341203 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1415/2816] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:50:14 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:50:14.342677 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ce1b9-a2a0-4fc9-8117-2a14deefef5a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:14.355535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ce1b9-a2a0-4fc9-8117-2a14deefef5a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:14.408542 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:14.420474 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aae67e-99e7-4a15-a3a1-9eb693338e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:14.421135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aae67e-99e7-4a15-a3a1-9eb693338e95 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:14.421462 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aae67e-99e7-4a15-a3a1-9eb693338e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:14.421695 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aae67e-99e7-4a15-a3a1-9eb693338e95 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:14.422372 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aae67e-99e7-4a15-a3a1-9eb693338e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:14.424347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aae67e-99e7-4a15-a3a1-9eb693338e95 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:14.520437 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1402/2817] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:50:14 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:50:14.524593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b4920-f6d7-43a8-8837-76d5571c9608 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:14.535907 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b4920-f6d7-43a8-8837-76d5571c9608 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:14.540806 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf855-599c-42aa-9a59-a523028e53c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:14.541340 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf855-599c-42aa-9a59-a523028e53c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:14.541635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf855-599c-42aa-9a59-a523028e53c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:14.541914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf855-599c-42aa-9a59-a523028e53c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:14.542600 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edf855-599c-42aa-9a59-a523028e53c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:14.544295 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edf855-599c-42aa-9a59-a523028e53c4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:14.560174 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1416/2818] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:50:14.561452 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aae67e-99e7-4a15-a3a1-9eb693338e95 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:14.574822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aae67e-99e7-4a15-a3a1-9eb693338e95 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:14.582094 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd20eea-d040-4f1a-87d5-d14700cab799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:14.582679 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd20eea-d040-4f1a-87d5-d14700cab799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:14.583351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd20eea-d040-4f1a-87d5-d14700cab799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:14.583642 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd20eea-d040-4f1a-87d5-d14700cab799 admin admin] PATH_INFO: `/v3/projects/cb7fb091c64c4fafa14f731cb275942f/users/93f677f7e2fd4a439bcbd4af04908c4e/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:14.608536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd20eea-d040-4f1a-87d5-d14700cab799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7fb091c64c4fafa14f731cb275942f, user_id=93f677f7e2fd4a439bcbd4af04908c4e, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:14.609432 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd20eea-d040-4f1a-87d5-d14700cab799 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:14.736099 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1403/2819] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:50:14 2022] PUT /identity/v3/projects/cb7fb091c64c4fafa14f731cb275942f/users/93f677f7e2fd4a439bcbd4af04908c4e/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:14.736914 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf855-599c-42aa-9a59-a523028e53c4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:14.748623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf855-599c-42aa-9a59-a523028e53c4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:14.753860 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:14.805396 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7dbbd3-9b4c-4161-bbc8-d537bca60154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:14.806009 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dbbd3-9b4c-4161-bbc8-d537bca60154 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:14.806288 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dbbd3-9b4c-4161-bbc8-d537bca60154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:14.806638 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dbbd3-9b4c-4161-bbc8-d537bca60154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:14.807323 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7dbbd3-9b4c-4161-bbc8-d537bca60154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:14.808981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7dbbd3-9b4c-4161-bbc8-d537bca60154 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:14.818757 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1417/2820] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:50:14.823829 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd20eea-d040-4f1a-87d5-d14700cab799 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:14.839932 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd20eea-d040-4f1a-87d5-d14700cab799 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:14.842420 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59004d6c-0830-474a-8e12-ce53e6b45014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:14.844229 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59004d6c-0830-474a-8e12-ce53e6b45014 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:14.844668 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59004d6c-0830-474a-8e12-ce53e6b45014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:14.844954 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59004d6c-0830-474a-8e12-ce53e6b45014 admin admin] PATH_INFO: `/v3/projects/cb7fb091c64c4fafa14f731cb275942f/users/93f677f7e2fd4a439bcbd4af04908c4e/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:14.847272 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59004d6c-0830-474a-8e12-ce53e6b45014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7fb091c64c4fafa14f731cb275942f, user_id=93f677f7e2fd4a439bcbd4af04908c4e, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:14.849925 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59004d6c-0830-474a-8e12-ce53e6b45014 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:14.866956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1404/2821] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:50:14 2022] PUT /identity/v3/projects/cb7fb091c64c4fafa14f731cb275942f/users/93f677f7e2fd4a439bcbd4af04908c4e/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:21.735973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b9725-f28d-4c64-90c6-bc378bbb071f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:21.736635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b9725-f28d-4c64-90c6-bc378bbb071f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:21.736635 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b9725-f28d-4c64-90c6-bc378bbb071f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:21.769320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-463b9725-f28d-4c64-90c6-bc378bbb071f None None] MFA Rules not processed for user `93f677f7e2fd4a439bcbd4af04908c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:21.770952 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-463b9725-f28d-4c64-90c6-bc378bbb071f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:21.844908 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1418/2822] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:21.858478 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59004d6c-0830-474a-8e12-ce53e6b45014 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:21.860605 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-59004d6c-0830-474a-8e12-ce53e6b45014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:21.956186 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59004d6c-0830-474a-8e12-ce53e6b45014 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:21.964201 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91aec8-6c6e-40e6-95ab-464cf04783b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:21.964918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91aec8-6c6e-40e6-95ab-464cf04783b9 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:21.965261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91aec8-6c6e-40e6-95ab-464cf04783b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:21.965601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91aec8-6c6e-40e6-95ab-464cf04783b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:22.036367 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91aec8-6c6e-40e6-95ab-464cf04783b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:22.038074 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91aec8-6c6e-40e6-95ab-464cf04783b9 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:22.049538 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1405/2823] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:50:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:50:26.458183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e3298-9b74-4edf-a859-c1a0a1ea5b00 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:26.465791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e3298-9b74-4edf-a859-c1a0a1ea5b00 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:26.465791 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e3298-9b74-4edf-a859-c1a0a1ea5b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:26.477996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f25e3298-9b74-4edf-a859-c1a0a1ea5b00 None None] MFA Rules not processed for user `0aff8f9d5721484cb58bcc430e165b15`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:26.479188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f25e3298-9b74-4edf-a859-c1a0a1ea5b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:26.486824 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1419/2824] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:30.840733 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ed89d-1ab5-4a43-a69c-3b2a266355bd None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:30.841230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ed89d-1ab5-4a43-a69c-3b2a266355bd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:30.841411 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ed89d-1ab5-4a43-a69c-3b2a266355bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:30.872405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-915ed89d-1ab5-4a43-a69c-3b2a266355bd None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:30.873620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-915ed89d-1ab5-4a43-a69c-3b2a266355bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:30.905511 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:30.937663 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1406/2825] 10.222.0.64 () {58 vars in 967 bytes} [Thu Aug 4 07:50:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:31.061664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e3298-9b74-4edf-a859-c1a0a1ea5b00 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:31.063271 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f25e3298-9b74-4edf-a859-c1a0a1ea5b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:31.135291 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e3298-9b74-4edf-a859-c1a0a1ea5b00 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:31.140576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c92845-52fa-49cb-af6d-e1d3df4dd40c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:31.141207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c92845-52fa-49cb-af6d-e1d3df4dd40c service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:31.141491 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c92845-52fa-49cb-af6d-e1d3df4dd40c service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:31.141769 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c92845-52fa-49cb-af6d-e1d3df4dd40c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:31.150633 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c92845-52fa-49cb-af6d-e1d3df4dd40c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:31.152258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c92845-52fa-49cb-af6d-e1d3df4dd40c service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:31.161003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1420/2826] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:50:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 07:50:32.292682 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0afb2-ccb5-4a3c-982d-fd7798528534 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:32.293164 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0afb2-ccb5-4a3c-982d-fd7798528534 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:32.293260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0afb2-ccb5-4a3c-982d-fd7798528534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:32.315081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-67b0afb2-ccb5-4a3c-982d-fd7798528534 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:32.316360 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-67b0afb2-ccb5-4a3c-982d-fd7798528534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:32.323725 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1407/2827] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:50:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:32.336775 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c92845-52fa-49cb-af6d-e1d3df4dd40c service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:32.348417 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c92845-52fa-49cb-af6d-e1d3df4dd40c service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:32.353480 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a8b56-b1a2-4f2e-8127-6ae6c0fee126 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:32.354294 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a8b56-b1a2-4f2e-8127-6ae6c0fee126 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:32.356530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a8b56-b1a2-4f2e-8127-6ae6c0fee126 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:32.356530 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a8b56-b1a2-4f2e-8127-6ae6c0fee126 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:32.364784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a8b56-b1a2-4f2e-8127-6ae6c0fee126 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:32.366388 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a8b56-b1a2-4f2e-8127-6ae6c0fee126 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:32.371090 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:32.382170 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1421/2828] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:50:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:50:33.380985 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a919a08-715a-4608-ac71-4b36b5941e18 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:33.381535 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a919a08-715a-4608-ac71-4b36b5941e18 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:33.381837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a919a08-715a-4608-ac71-4b36b5941e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:33.403906 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6a919a08-715a-4608-ac71-4b36b5941e18 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:33.405514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6a919a08-715a-4608-ac71-4b36b5941e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:33.413683 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1408/2829] 10.222.0.64 () {58 vars in 967 bytes} [Thu Aug 4 07:50:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:33.427005 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a8b56-b1a2-4f2e-8127-6ae6c0fee126 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:33.438937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a8b56-b1a2-4f2e-8127-6ae6c0fee126 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:33.444646 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dd077-9660-4689-86c3-3192921e6ffd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:33.445283 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dd077-9660-4689-86c3-3192921e6ffd service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:33.445678 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dd077-9660-4689-86c3-3192921e6ffd service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:33.445955 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dd077-9660-4689-86c3-3192921e6ffd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:33.456142 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dd077-9660-4689-86c3-3192921e6ffd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:33.457816 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dd077-9660-4689-86c3-3192921e6ffd service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:33.466899 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1422/2830] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:50:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:50:34.087254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebbbb8-b614-424a-8157-05dab0f01c2e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:34.087546 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebbbb8-b614-424a-8157-05dab0f01c2e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:34.089840 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebbbb8-b614-424a-8157-05dab0f01c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:34.110825 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cbebbbb8-b614-424a-8157-05dab0f01c2e None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:34.112092 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cbebbbb8-b614-424a-8157-05dab0f01c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:34.125101 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1409/2831] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Aug 4 07:50:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:34.133086 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dd077-9660-4689-86c3-3192921e6ffd service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:34.151082 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dd077-9660-4689-86c3-3192921e6ffd service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:34.157280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4f07f-439b-4018-bac8-b2b40f4a7b77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:34.158014 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f07f-439b-4018-bac8-b2b40f4a7b77 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:34.158394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f07f-439b-4018-bac8-b2b40f4a7b77 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:34.158794 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f07f-439b-4018-bac8-b2b40f4a7b77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:34.171419 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4f07f-439b-4018-bac8-b2b40f4a7b77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:34.173349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4f07f-439b-4018-bac8-b2b40f4a7b77 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:34.188197 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1423/2832] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:50:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 07:50:34.875601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a782e-ac7b-4e7a-9bb0-d502436e19a5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:34.876562 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a782e-ac7b-4e7a-9bb0-d502436e19a5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:34.876775 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a782e-ac7b-4e7a-9bb0-d502436e19a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:34.896822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3a9a782e-ac7b-4e7a-9bb0-d502436e19a5 None None] MFA Rules not processed for user `0c30c2fd1c4648308fac6606526451ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:34.898043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3a9a782e-ac7b-4e7a-9bb0-d502436e19a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:34.906512 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1410/2833] 10.222.0.64 () {58 vars in 967 bytes} [Thu Aug 4 07:50:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:34.918494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4f07f-439b-4018-bac8-b2b40f4a7b77 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:34.930068 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4f07f-439b-4018-bac8-b2b40f4a7b77 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:34.935445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22e736-253a-4911-b7ca-77141127322b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9503e2888d5e4831afc0d9d9522c3eea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:34.936175 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22e736-253a-4911-b7ca-77141127322b service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:34.936457 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22e736-253a-4911-b7ca-77141127322b service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:34.936736 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22e736-253a-4911-b7ca-77141127322b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:34.945831 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22e736-253a-4911-b7ca-77141127322b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:34.947428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22e736-253a-4911-b7ca-77141127322b service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:34.956429 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1424/2834] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Aug 4 07:50:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 07:50:37.820961 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a782e-ac7b-4e7a-9bb0-d502436e19a5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:37.902822 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a782e-ac7b-4e7a-9bb0-d502436e19a5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:37.908173 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:37.909087 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:37.909378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:37.909664 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] PATH_INFO: `/v3/users/2d3d739c143c496f8ba2de5db919cbb5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:37.910883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d3d739c143c496f8ba2de5db919cbb5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:37.913078 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:38.098864 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d3d739c143c496f8ba2de5db919cbb5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:38.099584 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d3d739c143c496f8ba2de5db919cbb5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:38.100215 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d3d739c143c496f8ba2de5db919cbb5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:38.113270 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d3d739c143c496f8ba2de5db919cbb5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:38.164722 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d3d739c143c496f8ba2de5db919cbb5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:38.181861 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1411/2835] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:50:37 2022] DELETE /identity/v3/users/2d3d739c143c496f8ba2de5db919cbb5 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:38.187103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65c64e-b4b3-4970-830b-97dc56dc19a4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:38.187559 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65c64e-b4b3-4970-830b-97dc56dc19a4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:38.187771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65c64e-b4b3-4970-830b-97dc56dc19a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:38.207344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1d65c64e-b4b3-4970-830b-97dc56dc19a4 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:38.209052 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1d65c64e-b4b3-4970-830b-97dc56dc19a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:38.272544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1425/2836] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:38.283484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:38.285036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:38.318497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:38.347469 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f919c-2fca-4336-91e9-caab8094bf39 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:38.351649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88932b5f-1598-4435-867e-5636c6ad5c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:38.352527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88932b5f-1598-4435-867e-5636c6ad5c09 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:38.353025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88932b5f-1598-4435-867e-5636c6ad5c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:38.353837 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88932b5f-1598-4435-867e-5636c6ad5c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:38.364384 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88932b5f-1598-4435-867e-5636c6ad5c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:38.365886 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88932b5f-1598-4435-867e-5636c6ad5c09 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:38.377481 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1412/2837] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:50:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:50:38.715350 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65c64e-b4b3-4970-830b-97dc56dc19a4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:38.716867 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1d65c64e-b4b3-4970-830b-97dc56dc19a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:38.745920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65c64e-b4b3-4970-830b-97dc56dc19a4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:38.745920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:38.745920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:38.745920 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:38.746807 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] PATH_INFO: `/v3/projects/0ce7dff8f8e54febbd219ec2d9933213` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:38.747401 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce7dff8f8e54febbd219ec2d9933213)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:38.750428 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:38.890983 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce7dff8f8e54febbd219ec2d9933213', 'request_id': 'req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:38.993609 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce7dff8f8e54febbd219ec2d9933213', 'request_id': 'req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:38.999876 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce7dff8f8e54febbd219ec2d9933213', 'request_id': 'req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:39.011964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] The token cache is being invalidate because project 0ce7dff8f8e54febbd219ec2d9933213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:50:39.012284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:39.012939 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b7e826a4-cb43-4afe-9840-7ee6e371a0b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:39.015028 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1426/2838] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:50:38 2022] DELETE /identity/v3/projects/0ce7dff8f8e54febbd219ec2d9933213 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:39.020131 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0458-29db-4b7b-9254-9f6f3d4c2071 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:39.020575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0458-29db-4b7b-9254-9f6f3d4c2071 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:39.020575 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0458-29db-4b7b-9254-9f6f3d4c2071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:39.041555 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7b4f0458-29db-4b7b-9254-9f6f3d4c2071 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:39.042765 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7b4f0458-29db-4b7b-9254-9f6f3d4c2071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:39.108987 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1413/2839] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 07:50:39.116960 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc138cd-8585-4e8b-8394-6b26513c90b2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:39.117250 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc138cd-8585-4e8b-8394-6b26513c90b2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:39.117537 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc138cd-8585-4e8b-8394-6b26513c90b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:39.140473 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fdc138cd-8585-4e8b-8394-6b26513c90b2 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:39.141857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fdc138cd-8585-4e8b-8394-6b26513c90b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:39.151494 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1427/2840] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:39.157413 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0458-29db-4b7b-9254-9f6f3d4c2071 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:39.168658 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0458-29db-4b7b-9254-9f6f3d4c2071 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:39.174315 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7242dac-c389-4565-97b3-f61e6deb4809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:39.175043 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7242dac-c389-4565-97b3-f61e6deb4809 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:39.175328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7242dac-c389-4565-97b3-f61e6deb4809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:39.175723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7242dac-c389-4565-97b3-f61e6deb4809 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:39.176456 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7242dac-c389-4565-97b3-f61e6deb4809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:50:39.176947 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7242dac-c389-4565-97b3-f61e6deb4809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:39.181371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7242dac-c389-4565-97b3-f61e6deb4809 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:39.203036 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1414/2841] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:50:39.207477 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc138cd-8585-4e8b-8394-6b26513c90b2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:39.218046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc138cd-8585-4e8b-8394-6b26513c90b2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:39.230056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1d3d9-a0e8-428f-a410-b9c6eb24179a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:39.230702 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1d3d9-a0e8-428f-a410-b9c6eb24179a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:39.230987 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1d3d9-a0e8-428f-a410-b9c6eb24179a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:39.231258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1d3d9-a0e8-428f-a410-b9c6eb24179a admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:39.231947 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1d3d9-a0e8-428f-a410-b9c6eb24179a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:39.234110 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1d3d9-a0e8-428f-a410-b9c6eb24179a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:39.313584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1428/2842] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:50:39 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:50:39.315230 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7242dac-c389-4565-97b3-f61e6deb4809 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:39.326285 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7242dac-c389-4565-97b3-f61e6deb4809 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:39.402233 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09a606-fb54-4a3d-8ed1-d26e1ef60fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:39.403133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09a606-fb54-4a3d-8ed1-d26e1ef60fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:39.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09a606-fb54-4a3d-8ed1-d26e1ef60fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:39.403670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09a606-fb54-4a3d-8ed1-d26e1ef60fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:39.404634 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09a606-fb54-4a3d-8ed1-d26e1ef60fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:39.407433 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09a606-fb54-4a3d-8ed1-d26e1ef60fd3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:39.485021 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1415/2843] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:50:39 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:50:39.489120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1d3d9-a0e8-428f-a410-b9c6eb24179a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:39.501010 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1d3d9-a0e8-428f-a410-b9c6eb24179a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:39.505747 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65a481-c3fa-4f89-a3ec-5e0c217182d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:39.506523 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65a481-c3fa-4f89-a3ec-5e0c217182d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:39.506872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65a481-c3fa-4f89-a3ec-5e0c217182d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:39.507214 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65a481-c3fa-4f89-a3ec-5e0c217182d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:39.507976 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65a481-c3fa-4f89-a3ec-5e0c217182d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:39.509699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65a481-c3fa-4f89-a3ec-5e0c217182d1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:39.520803 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1429/2844] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:50:39.525800 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09a606-fb54-4a3d-8ed1-d26e1ef60fd3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:39.537986 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09a606-fb54-4a3d-8ed1-d26e1ef60fd3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:39.542888 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c83e34-c7d6-4e9b-9a3c-20ef482be441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:39.543586 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83e34-c7d6-4e9b-9a3c-20ef482be441 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:39.544081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83e34-c7d6-4e9b-9a3c-20ef482be441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:39.544425 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83e34-c7d6-4e9b-9a3c-20ef482be441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:39.545198 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c83e34-c7d6-4e9b-9a3c-20ef482be441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:39.547514 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c83e34-c7d6-4e9b-9a3c-20ef482be441 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:39.566190 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1416/2845] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:50:39.566447 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65a481-c3fa-4f89-a3ec-5e0c217182d1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:39.572390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:39.576387 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65a481-c3fa-4f89-a3ec-5e0c217182d1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:39.581624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab3d4f-85ce-4699-a0c7-a7866d941acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:39.582300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab3d4f-85ce-4699-a0c7-a7866d941acb admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:39.582587 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab3d4f-85ce-4699-a0c7-a7866d941acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:39.582891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab3d4f-85ce-4699-a0c7-a7866d941acb admin admin] PATH_INFO: `/v3/projects/6a46d89169484583a105cbc8cb37c339/users/de76bb3b7cae47688f1493cf6f79b495/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:39.598852 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab3d4f-85ce-4699-a0c7-a7866d941acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a46d89169484583a105cbc8cb37c339, user_id=de76bb3b7cae47688f1493cf6f79b495, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:39.601809 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab3d4f-85ce-4699-a0c7-a7866d941acb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:39.687311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1430/2846] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:50:39 2022] PUT /identity/v3/projects/6a46d89169484583a105cbc8cb37c339/users/de76bb3b7cae47688f1493cf6f79b495/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:39.698460 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc8bf-fa18-40cf-9da7-d0a07194ddbf None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:39.698749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc8bf-fa18-40cf-9da7-d0a07194ddbf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:39.699029 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc8bf-fa18-40cf-9da7-d0a07194ddbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:39.720724 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1ecbc8bf-fa18-40cf-9da7-d0a07194ddbf None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:39.721942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1ecbc8bf-fa18-40cf-9da7-d0a07194ddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:39.753346 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:39.783539 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1417/2847] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:39.795478 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab3d4f-85ce-4699-a0c7-a7866d941acb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:39.799938 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc8bf-fa18-40cf-9da7-d0a07194ddbf None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:39.800207 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-36ab3d4f-85ce-4699-a0c7-a7866d941acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:39.801497 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1ecbc8bf-fa18-40cf-9da7-d0a07194ddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:39.822745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc8bf-fa18-40cf-9da7-d0a07194ddbf None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:39.822745 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:39.823006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:39.823199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:39.823502 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] PATH_INFO: `/v3/users/2979498fe4614879800a313e1a932fc3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:39.824715 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2979498fe4614879800a313e1a932fc3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:39.827529 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:39.867393 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab3d4f-85ce-4699-a0c7-a7866d941acb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:39.875483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c5736-617a-4dfc-ba5f-f8e027c53af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:39.875483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c5736-617a-4dfc-ba5f-f8e027c53af1 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:39.875483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c5736-617a-4dfc-ba5f-f8e027c53af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:39.875483 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c5736-617a-4dfc-ba5f-f8e027c53af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:39.888178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c5736-617a-4dfc-ba5f-f8e027c53af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:39.888178 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c5736-617a-4dfc-ba5f-f8e027c53af1 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:39.895645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1431/2848] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:50:40.043403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2979498fe4614879800a313e1a932fc3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.044260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2979498fe4614879800a313e1a932fc3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.044870 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2979498fe4614879800a313e1a932fc3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.059577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2979498fe4614879800a313e1a932fc3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.135419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2979498fe4614879800a313e1a932fc3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.148776 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1418/2849] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:50:39 2022] DELETE /identity/v3/users/2979498fe4614879800a313e1a932fc3 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:50:40.446453 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c5736-617a-4dfc-ba5f-f8e027c53af1 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:40.459129 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c5736-617a-4dfc-ba5f-f8e027c53af1 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:40.524774 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:40.525422 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:40.525703 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:40.525988 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] PATH_INFO: `/v3/projects/78ce0150832b415abb997c2e26e5a007` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:40.527143 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ce0150832b415abb997c2e26e5a007)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:40.529093 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:40.651931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ce0150832b415abb997c2e26e5a007', 'request_id': 'req-71cd150c-41d2-4f23-9ec5-da3106cca5f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.719965 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ce0150832b415abb997c2e26e5a007', 'request_id': 'req-71cd150c-41d2-4f23-9ec5-da3106cca5f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.726425 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ce0150832b415abb997c2e26e5a007', 'request_id': 'req-71cd150c-41d2-4f23-9ec5-da3106cca5f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.738067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] The token cache is being invalidate because project 78ce0150832b415abb997c2e26e5a007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:50:40.738551 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.739303 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71cd150c-41d2-4f23-9ec5-da3106cca5f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.740911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1432/2850] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:50:40 2022] DELETE /identity/v3/projects/78ce0150832b415abb997c2e26e5a007 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:40.744883 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:40.746483 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:40.761940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a35aa9-ed33-4f52-aea7-65018db3513e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:40.822468 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:40.823116 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:40.823402 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:40.823684 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] PATH_INFO: `/v3/users/c5d08b60a0684fc180b17b37015c3865` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:40.824967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5d08b60a0684fc180b17b37015c3865)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:40.827505 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:40.997973 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5d08b60a0684fc180b17b37015c3865'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.999300 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5d08b60a0684fc180b17b37015c3865'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:40.999950 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5d08b60a0684fc180b17b37015c3865'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:41.002223 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:41.012874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5d08b60a0684fc180b17b37015c3865'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:41.080744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5d08b60a0684fc180b17b37015c3865'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:41.093933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1419/2851] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:50:40 2022] DELETE /identity/v3/users/c5d08b60a0684fc180b17b37015c3865 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 07:50:41.269270 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db871d9-394e-4d58-9747-c8063d37a159 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:41.269568 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db871d9-394e-4d58-9747-c8063d37a159 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:41.269846 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db871d9-394e-4d58-9747-c8063d37a159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:41.298843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:41.303003 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8db871d9-394e-4d58-9747-c8063d37a159 None None] MFA Rules not processed for user `de76bb3b7cae47688f1493cf6f79b495`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:41.304492 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8db871d9-394e-4d58-9747-c8063d37a159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:41.378234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:41.379758 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:41.384476 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1433/2852] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:41.395758 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db871d9-394e-4d58-9747-c8063d37a159 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:41.397164 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8db871d9-394e-4d58-9747-c8063d37a159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:41.441068 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea35b1b-1449-472a-b93e-bbcf3bfe5bbd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:41.444874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3554d4-b75b-475a-a650-f043a86ee484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:41.448126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3554d4-b75b-475a-a650-f043a86ee484 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:41.448126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3554d4-b75b-475a-a650-f043a86ee484 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:41.448126 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3554d4-b75b-475a-a650-f043a86ee484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:41.449380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1d3554d4-b75b-475a-a650-f043a86ee484 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:41.458072 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db871d9-394e-4d58-9747-c8063d37a159 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:41.460702 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3554d4-b75b-475a-a650-f043a86ee484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:41.462459 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3554d4-b75b-475a-a650-f043a86ee484 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:41.462566 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270c5a6-d863-4fb1-af63-c5f0ca45d03c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eab096d4ccae4bc09e6177afd85a55de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:41.463085 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c5a6-d863-4fb1-af63-c5f0ca45d03c service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:41.463372 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c5a6-d863-4fb1-af63-c5f0ca45d03c service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:41.463650 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c5a6-d863-4fb1-af63-c5f0ca45d03c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:41.472672 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270c5a6-d863-4fb1-af63-c5f0ca45d03c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:41.474159 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270c5a6-d863-4fb1-af63-c5f0ca45d03c service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:41.478957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1420/2853] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:50:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:50:41.478957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3554d4-b75b-475a-a650-f043a86ee484 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:50:41.478957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1d3554d4-b75b-475a-a650-f043a86ee484 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:41.484716 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1434/2854] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:50:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:50:41.555666 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3554d4-b75b-475a-a650-f043a86ee484 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:50:41.562817 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:50:41.563589 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:41.564103 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:41.564454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] PATH_INFO: `/v3/projects/1ac52420449143a39f8f377499804dc5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:41.565855 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ac52420449143a39f8f377499804dc5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:50:41.567836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:50:41.714633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ac52420449143a39f8f377499804dc5', 'request_id': 'req-163a101c-3cdd-44e7-8615-1950ad3900d2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:41.720957 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ac52420449143a39f8f377499804dc5', 'request_id': 'req-163a101c-3cdd-44e7-8615-1950ad3900d2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:41.816379 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ac52420449143a39f8f377499804dc5', 'request_id': 'req-163a101c-3cdd-44e7-8615-1950ad3900d2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:41.828289 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] The token cache is being invalidate because project 1ac52420449143a39f8f377499804dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:50:41.828956 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:41.829638 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:50:41.831183 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1421/2855] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:50:41 2022] DELETE /identity/v3/projects/1ac52420449143a39f8f377499804dc5 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:50:42.107744 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:50:43.301657 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a897-e6f6-4534-8d8f-8392c1521f7e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:50:43.302168 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a897-e6f6-4534-8d8f-8392c1521f7e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:50:43.302265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a897-e6f6-4534-8d8f-8392c1521f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:50:43.321576 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1b86a897-e6f6-4534-8d8f-8392c1521f7e None None] MFA Rules not processed for user `de76bb3b7cae47688f1493cf6f79b495`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:50:43.322865 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1b86a897-e6f6-4534-8d8f-8392c1521f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:50:43.381565 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1435/2856] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:50:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:50:53.382298 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=77 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:51:01.308458 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:51:01.309991 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:51:01.369205 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a101c-3cdd-44e7-8615-1950ad3900d2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:51:01.373174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a11831-894a-4914-901f-dddadc1d57a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:51:01.373810 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a11831-894a-4914-901f-dddadc1d57a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:51:01.374097 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a11831-894a-4914-901f-dddadc1d57a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:51:01.374404 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a11831-894a-4914-901f-dddadc1d57a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:51:01.376398 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d5a11831-894a-4914-901f-dddadc1d57a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:51:01.438818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a11831-894a-4914-901f-dddadc1d57a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:51:01.440288 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a11831-894a-4914-901f-dddadc1d57a4 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:51:01.449969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1422/2857] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:51:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:51:11.445403 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=50 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:51:40.032934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a897-e6f6-4534-8d8f-8392c1521f7e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:51:40.042822 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a897-e6f6-4534-8d8f-8392c1521f7e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:51:40.046456 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78be163-e556-4d89-9399-f5a5ae40d26a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:51:40.047056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78be163-e556-4d89-9399-f5a5ae40d26a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:51:40.047332 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78be163-e556-4d89-9399-f5a5ae40d26a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:51:40.047603 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78be163-e556-4d89-9399-f5a5ae40d26a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:51:40.050163 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a78be163-e556-4d89-9399-f5a5ae40d26a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:51:40.119940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78be163-e556-4d89-9399-f5a5ae40d26a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:51:40.121406 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78be163-e556-4d89-9399-f5a5ae40d26a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:51:40.132706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1436/2858] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:51:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:51:42.480124 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a11831-894a-4914-901f-dddadc1d57a4 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:51:42.489917 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a11831-894a-4914-901f-dddadc1d57a4 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:51:42.495466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42e6d4-9d67-4321-992a-0bf93cf86134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:51:42.495466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42e6d4-9d67-4321-992a-0bf93cf86134 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:51:42.495466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42e6d4-9d67-4321-992a-0bf93cf86134 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:51:42.495466 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42e6d4-9d67-4321-992a-0bf93cf86134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:51:42.497056 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fd42e6d4-9d67-4321-992a-0bf93cf86134 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:51:42.515550 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42e6d4-9d67-4321-992a-0bf93cf86134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:51:42.517842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42e6d4-9d67-4321-992a-0bf93cf86134 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:51:42.529327 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1423/2859] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:51:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:51:50.127664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=34 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:51:52.524335 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:51:53.962987 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156fe18-c497-4b8a-aec1-5a520578a9e8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:51:53.963398 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156fe18-c497-4b8a-aec1-5a520578a9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:51:53.963517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156fe18-c497-4b8a-aec1-5a520578a9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:51:53.982318 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3156fe18-c497-4b8a-aec1-5a520578a9e8 None None] MFA Rules not processed for user `de76bb3b7cae47688f1493cf6f79b495`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:51:53.983445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3156fe18-c497-4b8a-aec1-5a520578a9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:51:53.990771 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1437/2860] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:51:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:51:54.628023 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42e6d4-9d67-4321-992a-0bf93cf86134 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:51:54.631363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fd42e6d4-9d67-4321-992a-0bf93cf86134 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:51:54.703383 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42e6d4-9d67-4321-992a-0bf93cf86134 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:51:54.709180 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069069b0-2aaa-4973-b5e4-d139f954b75a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:51:54.709799 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069069b0-2aaa-4973-b5e4-d139f954b75a service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:51:54.710083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069069b0-2aaa-4973-b5e4-d139f954b75a service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:51:54.710361 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069069b0-2aaa-4973-b5e4-d139f954b75a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:51:54.712161 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-069069b0-2aaa-4973-b5e4-d139f954b75a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:51:54.724147 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069069b0-2aaa-4973-b5e4-d139f954b75a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:51:54.725718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069069b0-2aaa-4973-b5e4-d139f954b75a service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:51:54.735603 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1424/2861] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:51:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:52:03.976973 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:52:04.731352 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=32 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:52:05.007937 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156fe18-c497-4b8a-aec1-5a520578a9e8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:52:05.018572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156fe18-c497-4b8a-aec1-5a520578a9e8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:52:05.022199 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6527a7e-6076-4ad5-9423-25ff27d7e7f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:52:05.022802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6527a7e-6076-4ad5-9423-25ff27d7e7f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:52:05.023078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6527a7e-6076-4ad5-9423-25ff27d7e7f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:52:05.023349 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6527a7e-6076-4ad5-9423-25ff27d7e7f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:52:05.025107 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f6527a7e-6076-4ad5-9423-25ff27d7e7f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:52:05.036071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6527a7e-6076-4ad5-9423-25ff27d7e7f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:52:05.037969 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6527a7e-6076-4ad5-9423-25ff27d7e7f6 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:52:05.047345 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1438/2862] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:52:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:52:15.042709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:52:59.593969 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069069b0-2aaa-4973-b5e4-d139f954b75a service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:52:59.605597 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069069b0-2aaa-4973-b5e4-d139f954b75a service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:52:59.611000 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38523150-db00-406f-9828-da3793a66dbf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:52:59.611649 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38523150-db00-406f-9828-da3793a66dbf service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:52:59.611936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38523150-db00-406f-9828-da3793a66dbf service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:52:59.612252 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38523150-db00-406f-9828-da3793a66dbf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:52:59.614083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-38523150-db00-406f-9828-da3793a66dbf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:52:59.626259 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38523150-db00-406f-9828-da3793a66dbf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:52:59.627951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38523150-db00-406f-9828-da3793a66dbf service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:52:59.638940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1425/2863] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:52:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:52:59.713690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6527a7e-6076-4ad5-9423-25ff27d7e7f6 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:52:59.715194 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f6527a7e-6076-4ad5-9423-25ff27d7e7f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:52:59.781690 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6527a7e-6076-4ad5-9423-25ff27d7e7f6 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:52:59.786484 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:52:59.787213 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:52:59.787504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:52:59.787782 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] PATH_INFO: `/v3/users/de76bb3b7cae47688f1493cf6f79b495` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:52:59.788996 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] RBAC: Authorizing `identity:delete_user(user_id=de76bb3b7cae47688f1493cf6f79b495)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:52:59.790912 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:52:59.913870 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de76bb3b7cae47688f1493cf6f79b495'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:52:59.914445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de76bb3b7cae47688f1493cf6f79b495'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:52:59.921337 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de76bb3b7cae47688f1493cf6f79b495'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:52:59.933206 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de76bb3b7cae47688f1493cf6f79b495'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:53:00.003682 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de76bb3b7cae47688f1493cf6f79b495'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:53:00.010789 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1439/2864] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:52:59 2022] DELETE /identity/v3/users/de76bb3b7cae47688f1493cf6f79b495 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:53:00.254440 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38523150-db00-406f-9828-da3793a66dbf service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:53:00.267323 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38523150-db00-406f-9828-da3793a66dbf service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:53:00.325578 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:53:00.326207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:53:00.326494 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:53:00.326777 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] PATH_INFO: `/v3/projects/6a46d89169484583a105cbc8cb37c339` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:53:00.328207 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a46d89169484583a105cbc8cb37c339)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:53:00.330114 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:53:00.408534 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a46d89169484583a105cbc8cb37c339', 'request_id': 'req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:53:00.415699 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a46d89169484583a105cbc8cb37c339', 'request_id': 'req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:53:00.471671 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a46d89169484583a105cbc8cb37c339', 'request_id': 'req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:53:00.482970 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] The token cache is being invalidate because project 6a46d89169484583a105cbc8cb37c339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:53:00.483286 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:53:00.483925 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:53:00.485471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1426/2865] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:53:00 2022] DELETE /identity/v3/projects/6a46d89169484583a105cbc8cb37c339 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:53:10.007301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:53:10.479867 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:53:29.724773 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:53:29.726276 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:53:29.784593 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4eec63-ee1c-4f07-8267-77015139804d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:53:29.832362 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e20114-c57b-4c24-a084-379564ba049b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:53:29.832904 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e20114-c57b-4c24-a084-379564ba049b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:53:29.833188 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e20114-c57b-4c24-a084-379564ba049b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:53:29.833463 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e20114-c57b-4c24-a084-379564ba049b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:53:29.835407 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-94e20114-c57b-4c24-a084-379564ba049b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:53:29.844464 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e20114-c57b-4c24-a084-379564ba049b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:53:29.845891 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e20114-c57b-4c24-a084-379564ba049b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:53:29.853970 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1440/2866] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:53:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:53:30.953757 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:53:30.956246 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:53:30.969949 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73a314-6e8e-4914-8379-5f5dc6fde5b4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:53:30.973723 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cc929-d92b-47cf-9181-3e80ac180761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:53:30.974350 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cc929-d92b-47cf-9181-3e80ac180761 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:53:30.974661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cc929-d92b-47cf-9181-3e80ac180761 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:53:30.974940 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cc929-d92b-47cf-9181-3e80ac180761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:53:30.977040 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-964cc929-d92b-47cf-9181-3e80ac180761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:53:31.001412 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e20114-c57b-4c24-a084-379564ba049b service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:53:31.002953 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-94e20114-c57b-4c24-a084-379564ba049b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:53:31.052199 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cc929-d92b-47cf-9181-3e80ac180761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:53:31.053718 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cc929-d92b-47cf-9181-3e80ac180761 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:53:31.065234 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1427/2867] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:53:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:53:31.085998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e20114-c57b-4c24-a084-379564ba049b service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:53:31.091928 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c20371-07ad-42fc-b0f7-144eb07265b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:53:31.092667 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c20371-07ad-42fc-b0f7-144eb07265b4 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:53:31.093059 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c20371-07ad-42fc-b0f7-144eb07265b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:53:31.093472 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c20371-07ad-42fc-b0f7-144eb07265b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:53:31.095471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-04c20371-07ad-42fc-b0f7-144eb07265b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:53:31.109539 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c20371-07ad-42fc-b0f7-144eb07265b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:53:31.111321 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c20371-07ad-42fc-b0f7-144eb07265b4 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:53:31.127361 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1441/2868] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:53:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 07:53:41.059942 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=34 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:53:41.120712 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=80 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:54:57.751566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cc929-d92b-47cf-9181-3e80ac180761 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:54:57.753012 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-964cc929-d92b-47cf-9181-3e80ac180761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:54:57.821341 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cc929-d92b-47cf-9181-3e80ac180761 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:54:57.826318 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b912a-5909-45b8-a08b-6cd2509fd36a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:54:57.827184 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b912a-5909-45b8-a08b-6cd2509fd36a service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:54:57.827471 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b912a-5909-45b8-a08b-6cd2509fd36a service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:54:57.827748 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b912a-5909-45b8-a08b-6cd2509fd36a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:54:57.829561 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c66b912a-5909-45b8-a08b-6cd2509fd36a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:54:57.841847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b912a-5909-45b8-a08b-6cd2509fd36a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:54:57.843435 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b912a-5909-45b8-a08b-6cd2509fd36a service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:54:57.854119 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1428/2869] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:54:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:55:07.849013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=32 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:55:09.413535 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c20371-07ad-42fc-b0f7-144eb07265b4 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:09.415950 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-04c20371-07ad-42fc-b0f7-144eb07265b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:09.429325 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c20371-07ad-42fc-b0f7-144eb07265b4 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:09.438277 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bf596-c51e-43cc-9d87-682eb79ca7ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:09.439038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf596-c51e-43cc-9d87-682eb79ca7ae service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:09.439423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf596-c51e-43cc-9d87-682eb79ca7ae service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:09.439821 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf596-c51e-43cc-9d87-682eb79ca7ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:09.441940 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fc3bf596-c51e-43cc-9d87-682eb79ca7ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:09.464196 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bf596-c51e-43cc-9d87-682eb79ca7ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:09.465864 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bf596-c51e-43cc-9d87-682eb79ca7ae service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:09.488105 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1442/2870] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:55:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:55:11.880005 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b912a-5909-45b8-a08b-6cd2509fd36a service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:11.895445 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b912a-5909-45b8-a08b-6cd2509fd36a service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:11.901968 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937e984-8177-4f04-bbc5-d9028ee5c226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:11.911574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937e984-8177-4f04-bbc5-d9028ee5c226 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:11.911574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937e984-8177-4f04-bbc5-d9028ee5c226 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:11.911574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937e984-8177-4f04-bbc5-d9028ee5c226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:11.911574 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f937e984-8177-4f04-bbc5-d9028ee5c226 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:11.918801 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937e984-8177-4f04-bbc5-d9028ee5c226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:11.921473 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937e984-8177-4f04-bbc5-d9028ee5c226 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:11.936157 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1429/2871] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:55:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:55:14.761995 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bf596-c51e-43cc-9d87-682eb79ca7ae service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:14.763445 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fc3bf596-c51e-43cc-9d87-682eb79ca7ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:14.776020 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bf596-c51e-43cc-9d87-682eb79ca7ae service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:14.779687 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484fb8c-203c-471f-8098-2728b355b6d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:14.780344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484fb8c-203c-471f-8098-2728b355b6d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:14.780631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484fb8c-203c-471f-8098-2728b355b6d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:14.780926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484fb8c-203c-471f-8098-2728b355b6d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:14.782830 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9484fb8c-203c-471f-8098-2728b355b6d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:14.846165 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484fb8c-203c-471f-8098-2728b355b6d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:14.847628 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484fb8c-203c-471f-8098-2728b355b6d6 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:14.857390 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1443/2872] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:55:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:55:18.454200 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937e984-8177-4f04-bbc5-d9028ee5c226 service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:18.466025 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937e984-8177-4f04-bbc5-d9028ee5c226 service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:18.471328 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b553b2-24bc-4c0b-af84-c2d91d3baeb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:18.472091 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b553b2-24bc-4c0b-af84-c2d91d3baeb3 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:18.472370 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b553b2-24bc-4c0b-af84-c2d91d3baeb3 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:18.472646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b553b2-24bc-4c0b-af84-c2d91d3baeb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:18.475095 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-16b553b2-24bc-4c0b-af84-c2d91d3baeb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:18.487782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b553b2-24bc-4c0b-af84-c2d91d3baeb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:18.489613 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b553b2-24bc-4c0b-af84-c2d91d3baeb3 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:18.501249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1430/2873] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:55:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:55:22.169577 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484fb8c-203c-471f-8098-2728b355b6d6 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:22.170981 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9484fb8c-203c-471f-8098-2728b355b6d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:22.183289 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484fb8c-203c-471f-8098-2728b355b6d6 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:22.187674 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:22.191115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:22.191115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:22.191115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] PATH_INFO: `/v3/users/0aff8f9d5721484cb58bcc430e165b15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:22.191115 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aff8f9d5721484cb58bcc430e165b15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:22.192339 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:22.420496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aff8f9d5721484cb58bcc430e165b15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:22.426041 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aff8f9d5721484cb58bcc430e165b15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:22.429493 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aff8f9d5721484cb58bcc430e165b15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:22.442818 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aff8f9d5721484cb58bcc430e165b15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:22.556916 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aff8f9d5721484cb58bcc430e165b15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:22.563219 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1444/2874] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:55:22 2022] DELETE /identity/v3/users/0aff8f9d5721484cb58bcc430e165b15 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:55:22.815523 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b553b2-24bc-4c0b-af84-c2d91d3baeb3 service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:22.827192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b553b2-24bc-4c0b-af84-c2d91d3baeb3 service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:22.891172 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:22.891736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:22.892046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:22.892326 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] PATH_INFO: `/v3/projects/541d2c7940b249fa822e6fcb5e433149` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:22.893484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=541d2c7940b249fa822e6fcb5e433149)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:22.895381 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:22.998449 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '541d2c7940b249fa822e6fcb5e433149', 'request_id': 'req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.004749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '541d2c7940b249fa822e6fcb5e433149', 'request_id': 'req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.100039 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '541d2c7940b249fa822e6fcb5e433149', 'request_id': 'req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.110793 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] The token cache is being invalidate because project 541d2c7940b249fa822e6fcb5e433149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:55:23.111081 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.111705 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.113221 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1431/2875] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:55:22 2022] DELETE /identity/v3/projects/541d2c7940b249fa822e6fcb5e433149 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:55:23.117895 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:23.119315 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:23.132449 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf0b38-3f00-408c-86b8-7e9b952454df admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:23.180592 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:55:23.189279 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944e8c4-a008-4396-896b-3ba3f66d156c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:23.189964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944e8c4-a008-4396-896b-3ba3f66d156c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:23.190248 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944e8c4-a008-4396-896b-3ba3f66d156c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:23.190526 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944e8c4-a008-4396-896b-3ba3f66d156c admin admin] PATH_INFO: `/v3/users/93f677f7e2fd4a439bcbd4af04908c4e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:23.191701 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944e8c4-a008-4396-896b-3ba3f66d156c admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f677f7e2fd4a439bcbd4af04908c4e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:23.193699 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944e8c4-a008-4396-896b-3ba3f66d156c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:23.342631 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3944e8c4-a008-4396-896b-3ba3f66d156c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f677f7e2fd4a439bcbd4af04908c4e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.343311 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3944e8c4-a008-4396-896b-3ba3f66d156c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f677f7e2fd4a439bcbd4af04908c4e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.350931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3944e8c4-a008-4396-896b-3ba3f66d156c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f677f7e2fd4a439bcbd4af04908c4e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.362561 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3944e8c4-a008-4396-896b-3ba3f66d156c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f677f7e2fd4a439bcbd4af04908c4e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.425467 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3944e8c4-a008-4396-896b-3ba3f66d156c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f677f7e2fd4a439bcbd4af04908c4e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.436074 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1445/2876] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 07:55:23 2022] DELETE /identity/v3/users/93f677f7e2fd4a439bcbd4af04908c4e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:55:23.689814 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:23.691254 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:23.706633 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafef273-bd93-4d74-8c9c-76fd3f40e1fd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:23.766955 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:23.767596 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:23.767882 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:23.768192 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] PATH_INFO: `/v3/projects/cb7fb091c64c4fafa14f731cb275942f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:23.769389 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb7fb091c64c4fafa14f731cb275942f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:23.771339 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:23.778282 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:55:23.888222 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb7fb091c64c4fafa14f731cb275942f', 'request_id': 'req-b3e29a71-a488-4fd0-a1fb-fb71827fec64'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.894877 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb7fb091c64c4fafa14f731cb275942f', 'request_id': 'req-b3e29a71-a488-4fd0-a1fb-fb71827fec64'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.968438 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb7fb091c64c4fafa14f731cb275942f', 'request_id': 'req-b3e29a71-a488-4fd0-a1fb-fb71827fec64'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.979296 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] The token cache is being invalidate because project cb7fb091c64c4fafa14f731cb275942f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 07:55:23.979641 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.980244 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b3e29a71-a488-4fd0-a1fb-fb71827fec64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 07:55:23.981779 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1432/2877] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 07:55:23 2022] DELETE /identity/v3/projects/cb7fb091c64c4fafa14f731cb275942f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:55:23.986706 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b03227-8fd7-43e5-bb8e-90e09e1b0351 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:23.986998 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b03227-8fd7-43e5-bb8e-90e09e1b0351 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:23.987307 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b03227-8fd7-43e5-bb8e-90e09e1b0351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:24.007958 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-20b03227-8fd7-43e5-bb8e-90e09e1b0351 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:55:24.009258 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-20b03227-8fd7-43e5-bb8e-90e09e1b0351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:24.068402 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1446/2878] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:55:24.077406 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da660fc-a402-4ad1-9978-8f8d6d6ccb1b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:24.077721 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da660fc-a402-4ad1-9978-8f8d6d6ccb1b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:24.078001 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da660fc-a402-4ad1-9978-8f8d6d6ccb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:24.097577 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1da660fc-a402-4ad1-9978-8f8d6d6ccb1b None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:55:24.098782 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1da660fc-a402-4ad1-9978-8f8d6d6ccb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:24.107035 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1433/2879] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:55:24.111135 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b03227-8fd7-43e5-bb8e-90e09e1b0351 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:24.121737 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b03227-8fd7-43e5-bb8e-90e09e1b0351 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:24.126344 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181a70a-6bcd-4dd6-badb-47d918433e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:24.126951 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181a70a-6bcd-4dd6-badb-47d918433e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:24.127238 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181a70a-6bcd-4dd6-badb-47d918433e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:24.127514 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181a70a-6bcd-4dd6-badb-47d918433e01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:24.128245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181a70a-6bcd-4dd6-badb-47d918433e01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 07:55:24.128557 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181a70a-6bcd-4dd6-badb-47d918433e01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:24.130200 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181a70a-6bcd-4dd6-badb-47d918433e01 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:24.142383 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1447/2880] 10.222.0.64 () {58 vars in 1153 bytes} [Thu Aug 4 07:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 07:55:24.146601 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da660fc-a402-4ad1-9978-8f8d6d6ccb1b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:24.155874 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da660fc-a402-4ad1-9978-8f8d6d6ccb1b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:24.160526 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459d717-f5fd-4d86-9efc-8931e77b3c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:24.163133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d717-f5fd-4d86-9efc-8931e77b3c8f admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:24.163133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d717-f5fd-4d86-9efc-8931e77b3c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:24.163133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d717-f5fd-4d86-9efc-8931e77b3c8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:24.163133 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459d717-f5fd-4d86-9efc-8931e77b3c8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:24.164371 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459d717-f5fd-4d86-9efc-8931e77b3c8f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:24.256330 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1434/2881] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:55:24 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:55:24.261301 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181a70a-6bcd-4dd6-badb-47d918433e01 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:24.271544 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181a70a-6bcd-4dd6-badb-47d918433e01 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:24.326910 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a87ed-66ad-46f3-9f39-ee012048ee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:24.327517 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a87ed-66ad-46f3-9f39-ee012048ee92 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:24.327799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a87ed-66ad-46f3-9f39-ee012048ee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:24.328103 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a87ed-66ad-46f3-9f39-ee012048ee92 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:24.328760 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a87ed-66ad-46f3-9f39-ee012048ee92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:24.330664 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a87ed-66ad-46f3-9f39-ee012048ee92 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:24.338495 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:55:24.402369 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1448/2882] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:55:24 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:55:24.406454 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459d717-f5fd-4d86-9efc-8931e77b3c8f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:24.417419 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459d717-f5fd-4d86-9efc-8931e77b3c8f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:24.422082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81d2d1-9763-4741-a5eb-c4daa5d618b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:24.422717 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81d2d1-9763-4741-a5eb-c4daa5d618b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:24.423006 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81d2d1-9763-4741-a5eb-c4daa5d618b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:24.423287 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81d2d1-9763-4741-a5eb-c4daa5d618b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:24.423999 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81d2d1-9763-4741-a5eb-c4daa5d618b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:24.425661 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81d2d1-9763-4741-a5eb-c4daa5d618b4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:24.436313 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1435/2883] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:55:24.440434 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a87ed-66ad-46f3-9f39-ee012048ee92 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:24.450745 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a87ed-66ad-46f3-9f39-ee012048ee92 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:24.456709 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e049acf-14d5-47d6-8b6a-c53c971e49c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:24.457320 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e049acf-14d5-47d6-8b6a-c53c971e49c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:24.457605 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e049acf-14d5-47d6-8b6a-c53c971e49c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:24.457913 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e049acf-14d5-47d6-8b6a-c53c971e49c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:24.458601 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e049acf-14d5-47d6-8b6a-c53c971e49c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:24.460575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e049acf-14d5-47d6-8b6a-c53c971e49c3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:24.471590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1449/2884] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:55:24.475697 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81d2d1-9763-4741-a5eb-c4daa5d618b4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:24.486194 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81d2d1-9763-4741-a5eb-c4daa5d618b4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:24.490791 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7ec58-5f7f-4e85-a796-ce79a248de61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:24.492136 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7ec58-5f7f-4e85-a796-ce79a248de61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:24.492423 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7ec58-5f7f-4e85-a796-ce79a248de61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:24.492703 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7ec58-5f7f-4e85-a796-ce79a248de61 admin admin] PATH_INFO: `/v3/projects/864edcc293034ccf875e9583868b33e0/users/7dc961b721be44b58c312819a0e42a15/roles/525e499f44594268b4f362c62bd35e00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:24.506967 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7ec58-5f7f-4e85-a796-ce79a248de61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=864edcc293034ccf875e9583868b33e0, user_id=7dc961b721be44b58c312819a0e42a15, role_id=525e499f44594268b4f362c62bd35e00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:24.509803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7ec58-5f7f-4e85-a796-ce79a248de61 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:24.586727 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1436/2885] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:55:24 2022] PUT /identity/v3/projects/864edcc293034ccf875e9583868b33e0/users/7dc961b721be44b58c312819a0e42a15/roles/525e499f44594268b4f362c62bd35e00 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:55:24.591857 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec566d-089e-479d-a64c-faa651969cd3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:24.592257 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec566d-089e-479d-a64c-faa651969cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:24.592536 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec566d-089e-479d-a64c-faa651969cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:24.611423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8aec566d-089e-479d-a64c-faa651969cd3 None None] MFA Rules not processed for user `79d0d7df7a094b5ba4b9a9bb3b449967`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:55:24.612660 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8aec566d-089e-479d-a64c-faa651969cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:24.669815 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1450/2886] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:55:24.679486 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7ec58-5f7f-4e85-a796-ce79a248de61 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:24.681216 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8ba7ec58-5f7f-4e85-a796-ce79a248de61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:24.738803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7ec58-5f7f-4e85-a796-ce79a248de61 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:24.742488 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702df77b-77cb-4cd8-a976-60af4eef20bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:24.743098 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702df77b-77cb-4cd8-a976-60af4eef20bf service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:24.743380 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702df77b-77cb-4cd8-a976-60af4eef20bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:24.743655 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702df77b-77cb-4cd8-a976-60af4eef20bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:24.752936 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702df77b-77cb-4cd8-a976-60af4eef20bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:24.754378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702df77b-77cb-4cd8-a976-60af4eef20bf service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:24.764017 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1437/2887] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:55:29.964147 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefecb52-b5d7-4996-801b-f778d86a846f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:29.964649 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefecb52-b5d7-4996-801b-f778d86a846f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:29.964753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefecb52-b5d7-4996-801b-f778d86a846f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:29.994427 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cefecb52-b5d7-4996-801b-f778d86a846f None None] MFA Rules not processed for user `7dc961b721be44b58c312819a0e42a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:55:29.995584 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cefecb52-b5d7-4996-801b-f778d86a846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:30.067554 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1451/2888] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:55:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:55:30.072405 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702df77b-77cb-4cd8-a976-60af4eef20bf service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:30.082818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702df77b-77cb-4cd8-a976-60af4eef20bf service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:30.088336 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f917d7-69ba-400a-b1d7-e9c091658940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:30.089053 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f917d7-69ba-400a-b1d7-e9c091658940 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:30.089430 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f917d7-69ba-400a-b1d7-e9c091658940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:30.089780 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f917d7-69ba-400a-b1d7-e9c091658940 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:30.090540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f917d7-69ba-400a-b1d7-e9c091658940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:30.092495 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f917d7-69ba-400a-b1d7-e9c091658940 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:30.187876 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1438/2889] 10.222.0.64 () {60 vars in 1153 bytes} [Thu Aug 4 07:55:30 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 07:55:30.191872 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefecb52-b5d7-4996-801b-f778d86a846f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:30.203513 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefecb52-b5d7-4996-801b-f778d86a846f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:30.264780 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d733b66-4cea-4860-a969-4491bcb2e9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:30.283284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d733b66-4cea-4860-a969-4491bcb2e9b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:30.283284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d733b66-4cea-4860-a969-4491bcb2e9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:30.283284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d733b66-4cea-4860-a969-4491bcb2e9b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:30.283284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d733b66-4cea-4860-a969-4491bcb2e9b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:30.283284 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d733b66-4cea-4860-a969-4491bcb2e9b7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:30.346450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:55:30.353352 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1452/2890] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Aug 4 07:55:30 2022] POST /identity/v3/users => generated 295 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 07:55:30.363625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f917d7-69ba-400a-b1d7-e9c091658940 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:30.363625 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:55:30.370368 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f917d7-69ba-400a-b1d7-e9c091658940 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:30.376089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecfb2d-1853-4024-b2ab-772d4ebbf29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:30.376089 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecfb2d-1853-4024-b2ab-772d4ebbf29a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:30.377897 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecfb2d-1853-4024-b2ab-772d4ebbf29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:30.378242 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecfb2d-1853-4024-b2ab-772d4ebbf29a admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:30.379008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ecfb2d-1853-4024-b2ab-772d4ebbf29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:30.382047 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ecfb2d-1853-4024-b2ab-772d4ebbf29a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:30.394014 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1439/2891] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:55:30.398953 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d733b66-4cea-4860-a969-4491bcb2e9b7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:30.409708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d733b66-4cea-4860-a969-4491bcb2e9b7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:30.414547 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74650134-30f9-4728-bbd4-d057f36f3908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:30.415222 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74650134-30f9-4728-bbd4-d057f36f3908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:30.415529 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74650134-30f9-4728-bbd4-d057f36f3908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:30.415792 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74650134-30f9-4728-bbd4-d057f36f3908 admin admin] PATH_INFO: `/v3/projects/3fb055b05d76417db5de0d4b292b43d6/users/f54578eee39b4b85aee7a9183a31fa3f/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:30.433795 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74650134-30f9-4728-bbd4-d057f36f3908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb055b05d76417db5de0d4b292b43d6, user_id=f54578eee39b4b85aee7a9183a31fa3f, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:30.437813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74650134-30f9-4728-bbd4-d057f36f3908 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:30.534675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1453/2892] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:55:30 2022] PUT /identity/v3/projects/3fb055b05d76417db5de0d4b292b43d6/users/f54578eee39b4b85aee7a9183a31fa3f/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:55:30.538670 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecfb2d-1853-4024-b2ab-772d4ebbf29a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:30.549220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecfb2d-1853-4024-b2ab-772d4ebbf29a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:30.608841 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe10dd-b2fa-463f-98b6-c29fa79850ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:30.609465 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe10dd-b2fa-463f-98b6-c29fa79850ba admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:30.609752 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe10dd-b2fa-463f-98b6-c29fa79850ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:30.610032 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe10dd-b2fa-463f-98b6-c29fa79850ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:30.610731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe10dd-b2fa-463f-98b6-c29fa79850ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:30.612472 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe10dd-b2fa-463f-98b6-c29fa79850ba admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:30.623863 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1440/2893] 10.222.0.64 () {58 vars in 1122 bytes} [Thu Aug 4 07:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 07:55:30.628169 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74650134-30f9-4728-bbd4-d057f36f3908 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:30.639095 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74650134-30f9-4728-bbd4-d057f36f3908 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:30.643693 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33474b74-9aed-416b-bd5b-3025a0cee767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:30.644375 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33474b74-9aed-416b-bd5b-3025a0cee767 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:30.644659 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33474b74-9aed-416b-bd5b-3025a0cee767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:30.644934 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33474b74-9aed-416b-bd5b-3025a0cee767 admin admin] PATH_INFO: `/v3/projects/3fb055b05d76417db5de0d4b292b43d6/users/f54578eee39b4b85aee7a9183a31fa3f/roles/74ea89d30ac5491ab1d8fa1c7284be2e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:30.647035 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33474b74-9aed-416b-bd5b-3025a0cee767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb055b05d76417db5de0d4b292b43d6, user_id=f54578eee39b4b85aee7a9183a31fa3f, role_id=74ea89d30ac5491ab1d8fa1c7284be2e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:30.649799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33474b74-9aed-416b-bd5b-3025a0cee767 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:30.659885 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1454/2894] 10.222.0.64 () {60 vars in 1483 bytes} [Thu Aug 4 07:55:30 2022] PUT /identity/v3/projects/3fb055b05d76417db5de0d4b292b43d6/users/f54578eee39b4b85aee7a9183a31fa3f/roles/74ea89d30ac5491ab1d8fa1c7284be2e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 07:55:36.486395 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84afac0-5b50-4e76-8972-813e1bbc38f5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:36.486900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84afac0-5b50-4e76-8972-813e1bbc38f5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:36.487013 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84afac0-5b50-4e76-8972-813e1bbc38f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:36.545217 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f84afac0-5b50-4e76-8972-813e1bbc38f5 None None] MFA Rules not processed for user `f54578eee39b4b85aee7a9183a31fa3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:55:36.546789 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f84afac0-5b50-4e76-8972-813e1bbc38f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:36.624805 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1441/2895] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:55:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:55:36.641826 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33474b74-9aed-416b-bd5b-3025a0cee767 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:36.649273 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-33474b74-9aed-416b-bd5b-3025a0cee767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:36.728262 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33474b74-9aed-416b-bd5b-3025a0cee767 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:36.734046 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c48d9-6726-4df1-aa04-8b2bc42a1273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:36.735622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c48d9-6726-4df1-aa04-8b2bc42a1273 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:36.735622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c48d9-6726-4df1-aa04-8b2bc42a1273 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:36.735622 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c48d9-6726-4df1-aa04-8b2bc42a1273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:36.795675 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c48d9-6726-4df1-aa04-8b2bc42a1273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:36.797245 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c48d9-6726-4df1-aa04-8b2bc42a1273 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:36.806512 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1455/2896] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:55:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:55:46.621842 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=67 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:55:46.802182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:55:57.086500 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05459db5-5361-4b65-93a8-72cce140b65c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:57.087003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05459db5-5361-4b65-93a8-72cce140b65c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:57.087122 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05459db5-5361-4b65-93a8-72cce140b65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:57.106716 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-05459db5-5361-4b65-93a8-72cce140b65c None None] MFA Rules not processed for user `7dc961b721be44b58c312819a0e42a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:55:57.107916 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-05459db5-5361-4b65-93a8-72cce140b65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:57.114781 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1442/2897] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:55:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:55:57.429441 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c48d9-6726-4df1-aa04-8b2bc42a1273 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:55:57.430985 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-809c48d9-6726-4df1-aa04-8b2bc42a1273 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:57.487590 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c48d9-6726-4df1-aa04-8b2bc42a1273 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:55:57.491265 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1e0a8-6820-4b0f-9446-87d39bb248af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:55:57.491875 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1e0a8-6820-4b0f-9446-87d39bb248af service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:55:57.492192 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1e0a8-6820-4b0f-9446-87d39bb248af service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:55:57.492509 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1e0a8-6820-4b0f-9446-87d39bb248af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:55:57.494296 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-59a1e0a8-6820-4b0f-9446-87d39bb248af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:55:57.504623 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1e0a8-6820-4b0f-9446-87d39bb248af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:55:57.506056 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1e0a8-6820-4b0f-9446-87d39bb248af service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:55:57.514720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1456/2898] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:55:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:56:07.101918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:56:07.510460 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=26 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:56:19.127220 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05459db5-5361-4b65-93a8-72cce140b65c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:56:19.136951 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05459db5-5361-4b65-93a8-72cce140b65c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:56:19.141082 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274b6fa-62e2-491d-a603-729a4ce426cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:56:19.141698 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274b6fa-62e2-491d-a603-729a4ce426cd service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:56:19.141982 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274b6fa-62e2-491d-a603-729a4ce426cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:56:19.142260 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274b6fa-62e2-491d-a603-729a4ce426cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:56:19.151218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274b6fa-62e2-491d-a603-729a4ce426cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:56:19.152736 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274b6fa-62e2-491d-a603-729a4ce426cd service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:56:19.162306 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1443/2899] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:56:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:56:29.158008 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:58:03.461926 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1e0a8-6820-4b0f-9446-87d39bb248af service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:58:03.470766 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1e0a8-6820-4b0f-9446-87d39bb248af service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:58:03.475146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab75aa-ce94-4d70-92d7-0d66d3379e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:58:03.475748 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab75aa-ce94-4d70-92d7-0d66d3379e71 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:58:03.476061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab75aa-ce94-4d70-92d7-0d66d3379e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:58:03.476343 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab75aa-ce94-4d70-92d7-0d66d3379e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:58:03.478071 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21ab75aa-ce94-4d70-92d7-0d66d3379e71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:58:03.487508 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab75aa-ce94-4d70-92d7-0d66d3379e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:58:03.488964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab75aa-ce94-4d70-92d7-0d66d3379e71 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:58:03.496978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1457/2900] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:58:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:58:13.493300 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:58:16.527856 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274b6fa-62e2-491d-a603-729a4ce426cd service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:58:16.537511 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274b6fa-62e2-491d-a603-729a4ce426cd service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:58:16.541363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359870b6-2e8e-4a74-bad8-7182dbb77732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:58:16.541975 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359870b6-2e8e-4a74-bad8-7182dbb77732 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:58:16.542275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359870b6-2e8e-4a74-bad8-7182dbb77732 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:58:16.542556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359870b6-2e8e-4a74-bad8-7182dbb77732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:58:16.544409 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-359870b6-2e8e-4a74-bad8-7182dbb77732 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:58:16.564566 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359870b6-2e8e-4a74-bad8-7182dbb77732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:58:16.566141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359870b6-2e8e-4a74-bad8-7182dbb77732 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:58:16.578347 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1444/2901] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:58:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:58:17.779721 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab75aa-ce94-4d70-92d7-0d66d3379e71 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:58:17.789373 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab75aa-ce94-4d70-92d7-0d66d3379e71 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:58:17.793027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ab133-7fc9-45fa-addf-2c658defa85e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:58:17.793632 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ab133-7fc9-45fa-addf-2c658defa85e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:58:17.793911 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ab133-7fc9-45fa-addf-2c658defa85e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:58:17.794183 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ab133-7fc9-45fa-addf-2c658defa85e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:58:17.796045 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c23ab133-7fc9-45fa-addf-2c658defa85e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:58:17.809204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ab133-7fc9-45fa-addf-2c658defa85e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:58:17.810618 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ab133-7fc9-45fa-addf-2c658defa85e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:58:17.821394 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1458/2902] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:58:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:58:21.258944 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7842b64-c25f-4f8e-b80c-f29004afd0c9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:58:21.259456 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7842b64-c25f-4f8e-b80c-f29004afd0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:58:21.259556 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7842b64-c25f-4f8e-b80c-f29004afd0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:58:21.278571 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a7842b64-c25f-4f8e-b80c-f29004afd0c9 None None] MFA Rules not processed for user `7dc961b721be44b58c312819a0e42a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 07:58:21.279749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a7842b64-c25f-4f8e-b80c-f29004afd0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:58:21.287310 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1445/2903] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 07:58:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 07:58:21.603359 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ab133-7fc9-45fa-addf-2c658defa85e service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:58:21.611901 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ab133-7fc9-45fa-addf-2c658defa85e service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:58:21.616204 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e086570-1cb2-4db1-90d4-7395580b4d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:58:21.616838 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e086570-1cb2-4db1-90d4-7395580b4d24 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:58:21.617126 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e086570-1cb2-4db1-90d4-7395580b4d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:58:21.617405 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e086570-1cb2-4db1-90d4-7395580b4d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:58:21.619123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2e086570-1cb2-4db1-90d4-7395580b4d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:58:21.628978 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e086570-1cb2-4db1-90d4-7395580b4d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:58:21.630399 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e086570-1cb2-4db1-90d4-7395580b4d24 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:58:21.638395 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1459/2904] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:58:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:58:22.416948 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7842b64-c25f-4f8e-b80c-f29004afd0c9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:58:22.418336 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a7842b64-c25f-4f8e-b80c-f29004afd0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:58:22.509218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7842b64-c25f-4f8e-b80c-f29004afd0c9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:58:22.514187 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8a4de-6420-4a65-af67-cf763a331ffd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:58:22.514803 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a4de-6420-4a65-af67-cf763a331ffd service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:58:22.515083 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a4de-6420-4a65-af67-cf763a331ffd service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:58:22.515355 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a4de-6420-4a65-af67-cf763a331ffd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:58:22.517135 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5de8a4de-6420-4a65-af67-cf763a331ffd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:58:22.529275 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8a4de-6420-4a65-af67-cf763a331ffd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:58:22.530847 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8a4de-6420-4a65-af67-cf763a331ffd service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:58:22.541153 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1446/2905] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:58:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:58:31.634784 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=12 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:58:32.536918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=41 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 07:58:36.748087 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e086570-1cb2-4db1-90d4-7395580b4d24 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 07:58:36.753964 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2e086570-1cb2-4db1-90d4-7395580b4d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:58:36.766102 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e086570-1cb2-4db1-90d4-7395580b4d24 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 07:58:36.772123 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c88168-9541-47e6-b0c6-da818b1345b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 07:58:36.772747 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c88168-9541-47e6-b0c6-da818b1345b9 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 07:58:36.773013 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c88168-9541-47e6-b0c6-da818b1345b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 07:58:36.773285 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c88168-9541-47e6-b0c6-da818b1345b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 07:58:36.774942 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-62c88168-9541-47e6-b0c6-da818b1345b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 07:58:36.788807 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c88168-9541-47e6-b0c6-da818b1345b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 07:58:36.790450 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c88168-9541-47e6-b0c6-da818b1345b9 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 07:58:36.802743 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1460/2906] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 07:58:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 07:58:46.797182 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 08:00:02.750187 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8a4de-6420-4a65-af67-cf763a331ffd service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 08:00:02.751741 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5de8a4de-6420-4a65-af67-cf763a331ffd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 08:00:02.840665 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8a4de-6420-4a65-af67-cf763a331ffd service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 08:00:02.846554 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd7d27-9870-42f8-9c88-abdee5a2e9c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 08:00:02.847169 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd7d27-9870-42f8-9c88-abdee5a2e9c7 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:00:02.847455 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd7d27-9870-42f8-9c88-abdee5a2e9c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:00:02.847731 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd7d27-9870-42f8-9c88-abdee5a2e9c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:00:02.849529 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-22bd7d27-9870-42f8-9c88-abdee5a2e9c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 08:00:02.861331 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd7d27-9870-42f8-9c88-abdee5a2e9c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 08:00:02.862892 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd7d27-9870-42f8-9c88-abdee5a2e9c7 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 08:00:02.872620 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1447/2907] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 08:00:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 08:00:12.868463 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=37 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 08:00:23.537589 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9501d-7e22-4a17-8ea8-e3ea04488754 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:00:23.538104 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9501d-7e22-4a17-8ea8-e3ea04488754 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:00:23.538223 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9501d-7e22-4a17-8ea8-e3ea04488754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:00:23.557488 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-43b9501d-7e22-4a17-8ea8-e3ea04488754 None None] MFA Rules not processed for user `7dc961b721be44b58c312819a0e42a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 08:00:23.558708 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-43b9501d-7e22-4a17-8ea8-e3ea04488754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 08:00:23.565572 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1461/2908] 10.222.0.64 () {58 vars in 958 bytes} [Thu Aug 4 08:00:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 08:00:28.148768 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd7d27-9870-42f8-9c88-abdee5a2e9c7 service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 08:00:28.179484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd7d27-9870-42f8-9c88-abdee5a2e9c7 service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 08:00:28.185996 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fb566-984a-4db9-a7a3-887b2e872449 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 08:00:28.186617 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fb566-984a-4db9-a7a3-887b2e872449 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:00:28.186904 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fb566-984a-4db9-a7a3-887b2e872449 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:00:28.187262 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fb566-984a-4db9-a7a3-887b2e872449 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:00:28.189129 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b95fb566-984a-4db9-a7a3-887b2e872449 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 08:00:28.215174 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fb566-984a-4db9-a7a3-887b2e872449 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 08:00:28.216798 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fb566-984a-4db9-a7a3-887b2e872449 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 08:00:28.228686 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1448/2909] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 08:00:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 08:00:33.552067 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 08:00:37.382531 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9501d-7e22-4a17-8ea8-e3ea04488754 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 08:00:37.395946 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9501d-7e22-4a17-8ea8-e3ea04488754 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 08:00:37.401894 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5cf5c-22b9-4f45-8f97-902e3222a8bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 08:00:37.402504 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5cf5c-22b9-4f45-8f97-902e3222a8bd service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:00:37.402787 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5cf5c-22b9-4f45-8f97-902e3222a8bd service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:00:37.403066 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5cf5c-22b9-4f45-8f97-902e3222a8bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:00:37.411191 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5cf5c-22b9-4f45-8f97-902e3222a8bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 08:00:37.412813 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5cf5c-22b9-4f45-8f97-902e3222a8bd service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 08:00:37.422552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1462/2910] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 08:00:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 08:00:38.222974 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=18 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 08:00:44.639927 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fb566-984a-4db9-a7a3-887b2e872449 service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 08:00:44.654046 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fb566-984a-4db9-a7a3-887b2e872449 service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 08:00:44.658900 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcf4ff-3ec3-4b18-8669-3163a6c4be80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eedd711c6114de1bfcdf692ce058794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 08:00:44.659536 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcf4ff-3ec3-4b18-8669-3163a6c4be80 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:00:44.659818 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcf4ff-3ec3-4b18-8669-3163a6c4be80 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:00:44.660544 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcf4ff-3ec3-4b18-8669-3163a6c4be80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:00:44.661977 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bcdcf4ff-3ec3-4b18-8669-3163a6c4be80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 08:00:44.676218 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcf4ff-3ec3-4b18-8669-3163a6c4be80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 08:00:44.683860 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcf4ff-3ec3-4b18-8669-3163a6c4be80 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 08:00:44.690093 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1449/2911] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 08:00:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 08:00:47.418497 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 08:00:54.685249 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 08:02:27.207065 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5cf5c-22b9-4f45-8f97-902e3222a8bd service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 08:02:27.227553 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5cf5c-22b9-4f45-8f97-902e3222a8bd service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 08:02:27.232496 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81df16-c8a2-4449-852c-d8872a5a0bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b80e701a81cb4fa893e55288559010f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 08:02:27.233109 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81df16-c8a2-4449-852c-d8872a5a0bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:02:27.233411 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81df16-c8a2-4449-852c-d8872a5a0bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:02:27.233685 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81df16-c8a2-4449-852c-d8872a5a0bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:02:27.305120 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81df16-c8a2-4449-852c-d8872a5a0bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 08:02:27.306588 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81df16-c8a2-4449-852c-d8872a5a0bf9 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 08:02:27.325987 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1463/2912] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 08:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 08:02:31.449181 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcf4ff-3ec3-4b18-8669-3163a6c4be80 service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 08:02:31.463334 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcf4ff-3ec3-4b18-8669-3163a6c4be80 service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 08:02:31.470225 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501ced1-cfc5-406e-9cab-dc94034676cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 08:02:31.471680 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501ced1-cfc5-406e-9cab-dc94034676cc service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:02:31.472257 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501ced1-cfc5-406e-9cab-dc94034676cc service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:02:31.473178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501ced1-cfc5-406e-9cab-dc94034676cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:02:31.475363 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2501ced1-cfc5-406e-9cab-dc94034676cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 08:02:31.488691 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501ced1-cfc5-406e-9cab-dc94034676cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 08:02:31.490567 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501ced1-cfc5-406e-9cab-dc94034676cc service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 08:02:31.501738 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1450/2913] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 08:02:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 08:02:35.185061 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81df16-c8a2-4449-852c-d8872a5a0bf9 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 08:02:35.197437 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81df16-c8a2-4449-852c-d8872a5a0bf9 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 08:02:35.202228 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 08:02:35.202867 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:02:35.203146 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:02:35.203423 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] PATH_INFO: `/v3/users/7dc961b721be44b58c312819a0e42a15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:02:35.204624 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc961b721be44b58c312819a0e42a15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 08:02:35.206578 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 08:02:35.363720 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc961b721be44b58c312819a0e42a15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:35.364501 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc961b721be44b58c312819a0e42a15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:35.371841 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc961b721be44b58c312819a0e42a15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:35.383777 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc961b721be44b58c312819a0e42a15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:35.457722 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc961b721be44b58c312819a0e42a15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:35.463931 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1464/2914] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 08:02:35 2022] DELETE /identity/v3/users/7dc961b721be44b58c312819a0e42a15 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 08:02:35.714527 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501ced1-cfc5-406e-9cab-dc94034676cc service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 08:02:35.728622 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501ced1-cfc5-406e-9cab-dc94034676cc service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 08:02:35.788918 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 08:02:35.789563 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:02:35.789879 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:02:35.790163 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] PATH_INFO: `/v3/projects/864edcc293034ccf875e9583868b33e0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:02:35.791349 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=864edcc293034ccf875e9583868b33e0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 08:02:35.793293 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 08:02:35.901749 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '864edcc293034ccf875e9583868b33e0', 'request_id': 'req-4ace8ef6-a03f-4627-a952-f4c17cbddae8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:35.908064 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '864edcc293034ccf875e9583868b33e0', 'request_id': 'req-4ace8ef6-a03f-4627-a952-f4c17cbddae8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:35.992261 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '864edcc293034ccf875e9583868b33e0', 'request_id': 'req-4ace8ef6-a03f-4627-a952-f4c17cbddae8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.016120 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] The token cache is being invalidate because project 864edcc293034ccf875e9583868b33e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 08:02:36.020236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.020236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.020236 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1451/2915] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 08:02:35 2022] DELETE /identity/v3/projects/864edcc293034ccf875e9583868b33e0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 08:02:36.023282 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 08:02:36.024801 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 08:02:36.038552 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fea3aa-c479-463a-959b-44b9f21f8dc9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 08:02:36.079208 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 08:02:36.098389 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 08:02:36.099038 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:02:36.099366 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:02:36.099645 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] PATH_INFO: `/v3/users/f54578eee39b4b85aee7a9183a31fa3f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:02:36.100874 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54578eee39b4b85aee7a9183a31fa3f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 08:02:36.102802 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 08:02:36.246497 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54578eee39b4b85aee7a9183a31fa3f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.247280 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54578eee39b4b85aee7a9183a31fa3f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.254511 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54578eee39b4b85aee7a9183a31fa3f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.266847 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54578eee39b4b85aee7a9183a31fa3f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.352078 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54578eee39b4b85aee7a9183a31fa3f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.358327 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1465/2916] 10.222.0.64 () {58 vars in 1224 bytes} [Thu Aug 4 08:02:36 2022] DELETE /identity/v3/users/f54578eee39b4b85aee7a9183a31fa3f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 08:02:36.661792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 08:02:36.661792 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 08:02:36.672178 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace8ef6-a03f-4627-a952-f4c17cbddae8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 08:02:36.734933 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d0d7df7a094b5ba4b9a9bb3b449967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60367118cabe491a9e4ea2e0f6945aa3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 08:02:36.735646 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:02:36.736003 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:02:36.736351 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] PATH_INFO: `/v3/projects/3fb055b05d76417db5de0d4b292b43d6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:02:36.737894 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fb055b05d76417db5de0d4b292b43d6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 08:02:36.740237 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 08:02:36.748689 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 08:02:36.862852 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fb055b05d76417db5de0d4b292b43d6', 'request_id': 'req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.869383 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fb055b05d76417db5de0d4b292b43d6', 'request_id': 'req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.943378 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fb055b05d76417db5de0d4b292b43d6', 'request_id': 'req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.955540 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] The token cache is being invalidate because project 3fb055b05d76417db5de0d4b292b43d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 08:02:36.955836 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.956657 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b4230ec2-3ccb-4bfc-9f3f-1d43c4e6eb4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 08:02:36.959141 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1452/2917] 10.222.0.64 () {58 vars in 1233 bytes} [Thu Aug 4 08:02:36 2022] DELETE /identity/v3/projects/3fb055b05d76417db5de0d4b292b43d6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 08:02:46.354715 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 08:02:46.967484 dv-cinder-849800-3-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 08:05:08.717799 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 08:05:08.719262 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 08:05:08.792260 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707c6c5-3393-4799-b63e-0af756243802 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 08:05:08.854843 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e942a9-1b78-41fd-85b0-1514a19afba0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'affaeff3cd294584b7238909e0457f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a29ed6718e0b46a69ba512b878ce710a', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 08:05:08.855471 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e942a9-1b78-41fd-85b0-1514a19afba0 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 08:05:08.855753 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e942a9-1b78-41fd-85b0-1514a19afba0 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 08:05:08.856057 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e942a9-1b78-41fd-85b0-1514a19afba0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 08:05:08.858027 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-48e942a9-1b78-41fd-85b0-1514a19afba0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 08:05:08.871575 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e942a9-1b78-41fd-85b0-1514a19afba0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 08:05:08.873209 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e942a9-1b78-41fd-85b0-1514a19afba0 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 08:05:08.884948 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1466/2918] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Aug 4 08:05:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 08:05:18.879714 dv-cinder-849800-3-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=58 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}